← 输入框|github.com|刷新
Skip to content

build(deps): bump dependency versions - #434

Merged
MthwRobinson merged 3 commits into
mainfrom
build/bumps-20240624
Jun 24, 2024
Merged

MthwRobinson merged 3 commits into
mainfrom
build/bumps-20240624

Conversation

@MthwRobinson

Copy link
Copy Markdown
Contributor

Summary

Bumps dependency versions for the API. Closes #432.

@awalker4 awalker4 left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Edit: oh, we should also bump the changelog to get a new build for quay

@MthwRobinson
MthwRobinson enabled auto-merge (squash) June 24, 2024 14:48
@MthwRobinson
MthwRobinson merged commit d5a878f into main Jun 24, 2024
@MthwRobinson
MthwRobinson deleted the build/bumps-20240624 branch June 24, 2024 14:59
tjtanaa added a commit to EmbeddedLLM/unstructured-api-executable that referenced this pull request Jul 11, 2024
…indows (#2)

## PR Summary
1. Merged changes from upstream.
2. Update `unstructuredio_api.spec`.
3. Update `unstructuredio_api.py`.
4. Add additional setup dependencies to the `docs/Windows.md`. 

* build(deps): version bumps for maintenance (Unstructured-IO#424)

### Summary
Version bumps for regular maintenance and to address moderate CVEs from
security scans.
- bump `unstructured` to `0.14.6`
- bump `unstructured-inference` to `0.7.35`

* build: replace rockylinux with chainguard/wolfi as a base image (Unstructured-IO#423)

### Summary
Updates the Dockerfile to use the Chainguard wolfi-base image to reduce
CVEs. Also adds a step in the docker publish job that scans the images
and checks for CVEs before publishing.

### Testing
Run `make docker-build` and  `make docker-start-api`, then try:
```
from unstructured.partition.api import partition_via_api

elements = partition_via_api(
    filename=filename,
    api_url="http://localhost:8000/general/v0/general",
    api_key="<API-KEY>",
    strategy="hi_res",
)

print("\n\n".join([str(el) for el in elements]))
```

* fix: build and push workflow failing due to missing `-f` option `buildx build` command (Unstructured-IO#425)

I noticed that images on main branch are failing to build (and push) due
to missing `-f` parameter in `docker buildx build`. By default it
expects `Dockerfile` to exist, but we only have `Dockerfile-amd64` and
`Dockerfile-arm64`


![image](https://github.com/Unstructured-IO/unstructured-api/assets/64484917/4527165a-909e-498d-b0ee-8bba4b1a13e4)

---------

Co-authored-by: christinestraub <christinemstraub@gmail.com>

* fix: update SHA for the base images (both architectures) after `base-images` repo update (Unstructured-IO#427)

build and publish CI steps are failing, because the base images have
changed in quay (their SHAs)

![image](https://github.com/Unstructured-IO/unstructured-api/assets/64484917/fc4e9aac-0820-4c90-9ad9-68cc6d9aad03)


![image](https://github.com/Unstructured-IO/unstructured-api/assets/64484917/fafe2ca4-dab2-4610-a26b-a7a4d56723a5)

* fix: revert to rockylinux SHA that works (arm64) (Unstructured-IO#428)

unnecessary SHA update introduced in
Unstructured-IO#427 that needs
to be reverted

* fix: re-add `DOCKER_IMAGE` env var in `Test image` step (Unstructured-IO#429)

shell syntax error occurs in docker-publish.yml workflow

* fix: invalid env var setting in `docker-publish` workflow (Unstructured-IO#430)

bug introduced in previous PR causing build failure on main

* fix: `docker-publish` workflow failing on main due to inexisting `ARCH` env var (Unstructured-IO#431)

* build(deps): bump dependency versions (Unstructured-IO#434)

### Summary

Bumps dependency versions for the API. Closes Unstructured-IO#432.

* fix/Fix MS Office filetype errors and harden docker smoketest (Unstructured-IO#436)

# Changes
**Fix for docx and other office files returning `{"detail":"File type
None is not supported."}`**
After moving to the wolfi base image, the `mimetypes` lib no longer
knows about these file extensions. To avoid issues like this, let's add
an explicit mapping for all the file extensions we care about. I added a
`filetypes.py` and moved `get_validated_mimetype` over. When this file
is imported, we'll call `mimetypes.add_type` for all file extensions we
support.

**Update smoke test coverage**
This bug snuck past because we were already providing the mimetype in
the docker smoke test. I updated `test_happy_path` to test against the
container with and without passing `content_type`. I added some missing
filetypes, and sorted the test params by extension so we can see when
new types are missing.

# Testing
The new smoke test will verify that all filetypes are working. You can
also `make docker-build && make docker-start-api`, and test out the docx
in the sample docs dir. On `main`, this file will give you the error
above.
```
curl 'http://localhost:8000/general/v0/general' \
--form 'files=@"fake.docx"'
```

* merge main; validated format: xml, txv, csv, xml, json, html, docs, docx, ppt, pptx, xlsx, xls, pdf

* compilable setting

* update Windows markdown

* Disable debug mode in `unstructuredio_api.spec`

* Enable pdf test case in `test_app.py`

---------

Co-authored-by: Christine Straub <christinemstraub@gmail.com>
Co-authored-by: Michał Martyniak <64484917+micmarty-deepsense@users.noreply.github.com>
Co-authored-by: Matt Robinson <mrobinson@unstructured.io>
Co-authored-by: Austin Walker <austin@unstructured.io>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
CyMule added a commit that referenced this pull request Sep 11, 2026
## What
Standalone API CI avoids repeated inference work and redundant image
transfers while retaining distinct smoke-test scenarios.

| Measurement | Before | After | Saved |
| --- | ---: | ---: | ---: |
| Total CI | 39m18s historical median | 26m15s | 13m03s (33.2%) |
| Smoke tests | 16m05s | 11m37s | 4m28s (27.8%) |
| Mode comparisons | ~14m35s | 5m07s | ~9m28s (64.9%) |

[Successful PR
run](https://github.com/Unstructured-IO/unstructured-api/actions/runs/34426279723).
Total baseline is nine historical runs; test-stage baselines come from
August 27. This is not a controlled comparison: runner variability and
faster individual inference calls contribute to the difference.

## Why
Some option checks unintentionally repeated default inference, gzip
parametrization duplicated seven cases, and the smoke-test client
installed unused dependencies. The test harness now sends the intended
options and avoids duplicated work. Release publishing also retags
images in the registry instead of pulling them only to retag them.

The 37 distinct smoke cases and seven mode comparisons remain covered.
Heavy PDF inference remains the largest test cost.

## Relationship to other work
Independent of Plugins, platform-libs, and shared-workflow #434/#437.
Core-product has its own vendored API and separate CI improvement in
#1660; its savings are not an additional stage of this pipeline.

Publishing speed has not been measured: that workflow runs only on main.
The timing table covers CI, not release publishing.

## Validation
Current GitHub checks pass, including the Docker build, 37 smoke tests,
seven mode comparisons, core tests, coverage, and lint. Six shell
regression tests and registry-retagging dry runs passed; publishing
itself still requires validation on main.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IndexError: list index out of range while extracting images from pdf?

2 participants