Repository navigation
Tags: Unstructured-IO/unstructured
Tags
chore: release 0.27.35 (#4606) Release PR: changes `unstructured/__version__.py` from `0.27.35-dev` to `0.27.35`, per CONTRIBUTING.md release-day step 1. No changelog edit (CHANGELOG.md is frozen). This is the first release since the post-release dev-bump automation landed (#4590). After the `0.27.35` GitHub Release is published, the workflow should open a `chore: start 0.27.36-dev after 0.27.35 release` PR automatically.
chore: release 0.27.34 (#4582) Drops the `-dev6` suffix so 0.27.34 can be tagged and published. There are no code changes. Since 0.27.33, this release contains: - Free decode buffers sooner when restoring original elements (#4524) - Free image buffers sooner during figure extraction (#4523) - Honor `encoding=` in `partition_json` and `partition_ndjson` (#4483) - Read a PowerPoint line break as a newline (#4571) - Drop the empty entry of an email address group (#4576) - Read CSV and TSV cells as the file holds them (#4574) - Classify text as an email address only when all of it is one (#4577) After merge, create the `0.27.34` GitHub release from the merge commit. That publishes to PyPI. <!-- This is an auto-generated description by cubic. --> <a href="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4582?utm_source=github" rel="nofollow">https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4582?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-dark.svg%2522%253E%253Csource" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-light.svg%2522%253E%253Cimg" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="View guided diff" src="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
fix(deps): bump certifi to 2026.10.19 (#4564) ## Why Eight eMudhra intermediate CA private keys were confirmed compromised ([article](https://thehackernews.com/2026/10/eight-emudhra-ca-keys-compromised.html)). certifi 2026.10.19 drops the **emSign ECC Root CA - G3** root. `requests` and `httpx` trust certifi's bundle and do not check revocation, so the older bundle accepts certificates chaining to that root. ## What - `uv.lock`: `certifi` upgraded to 2026.10.19 (`uv lock --upgrade-package certifi`; no other packages changed). - Version bumped to `0.27.33` with a CHANGELOG entry. Tests: lock consistency checked locally; full suites left to CI. Also requires `certifi>=2026.10.19` in `[project].dependencies`, so pip installs get the new bundle, not only the lockfile. **Release note:** this cuts `0.27.33` from main's `0.27.33-dev0`, so it also releases the chunking fix already on main (folded into the `0.27.33` CHANGELOG section). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated description by cubic. --> <a href="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4564?utm_source=github" rel="nofollow">https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4564?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-dark.svg%2522%253E%253Csource" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-light.svg%2522%253E%253Cimg" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="View guided diff" src="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
perf(xlsx): reduce dense-worksheet partitioning memory (#4563) ## What `partition_xlsx()` with subtable detection (the default) needs substantially less memory on dense worksheets, and the saving grows with sheet size. Output is unchanged. On synthetic dense sheets (a ten-column sheet split into three dense subtables, half integers, half short Unicode strings), peak RSS in a fresh process drops by roughly 40–60% depending on size, with comparable runtime. Working memory per populated cell drops to roughly a third. Serialized element JSON is byte-identical before and after. ## Why Two whole-sheet intermediates dominated memory on dense sheets: 1. **Subtable detection built a NetworkX graph** with a node per populated cell and an edge per neighbor pair. It now uses an iterative four-neighbor flood fill over populated cells (the approach from #4357), with the same component admission order, cell membership, row-overlap merging, table order and bounding boxes. 2. **Table HTML was rendered for the whole table at once** (pandas `to_html` string, its parsed DOM and a full-table token list). Rows are now formatted by pandas' formatter for the whole column, so float/date/string formatting is unchanged, and each row is rendered and parsed on its own before the next. `networkx` is no longer needed at runtime by the XLSX partitioner, so it moves from the `xlsx` extra to the test dependencies (it's still used by other extras that depend on it). ## Behavior - Same elements, text, `text_as_html` and metadata; exact-output tests cover mixed types, NaN/empty cells, wide floats that switch to scientific notation in another row, dates and HTML escaping. - The renderer uses pandas' private formatter classes, as the existing reader already does for `OpenpyxlReader`. Verified on pandas 2.1.4 (renderer-parity comparisons), 2.2.3 and 2.3.3. - The worksheet cell limit, empty-row compaction, empty-sheet handling and XLS padding handling are unchanged. ## Validation - XLSX and dense-memory suites pass. New tests compare component detection and rendered HTML against the previous implementation on generated sheets. - Ruff check/format pass. ## Release Sets the release to **0.27.32** (0.27.31 was taken by #4538, which merged first). It merges after #4419, #4516, #4517 and #4562, and its CHANGELOG collects their entries and this one into a single 0.27.32 section. #4524 and #4523 are not included and will go into a later release. Supersedes #4357. <!-- This is an auto-generated description by cubic. --> <a href="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4563?utm_source=github" rel="nofollow">https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4563?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-dark.svg%2522%253E%253Csource" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-light.svg%2522%253E%253Cimg" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="View guided diff" src="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
fix(email): keep non-ASCII sender and recipient names readable (#4543) ## What `partition_email()` now keeps non-ASCII sender and recipient names, and non-ASCII addresses, as written in the `sent_from`, `sent_to`, `cc_recipient` and `bcc_recipient` metadata. Closes #4542. ``` From: José García <jose@ejemplo.es> (RFC 2047-encoded on the wire, as mail clients write it) To: 张伟 <zhang@example.cn> Before: sent_from=['=?utf-8?b?Sm9zw6kgR2FyY8OtYQ==?= <jose@ejemplo.es>'] sent_to=['=?utf-8?b?5byg5Lyf?= <zhang@example.cn>'] After: sent_from=['José García <jose@ejemplo.es>'] sent_to=['张伟 <zhang@example.cn>'] From: José <josé@ejemplo.es> (RFC 6532 address) Before: UnicodeEncodeError, no elements After: sent_from=['José <josé@ejemplo.es>'] ``` ## Why The headers are already decoded (`email.policy.default`), but the four address properties formatted them with `email.utils.formataddr()`. It writes a non-ASCII display name as an RFC 2047 encoded-word, and it raises `UnicodeEncodeError` on a non-ASCII address. They now use `_format_address()`, which quotes and escapes the display name with the same rules as `formataddr()` but leaves non-ASCII text as is. For ASCII names and addresses the result is unchanged: on 50,000 random ASCII name/address pairs it matched `formataddr()` exactly. ## Validation - `test_partition_email_keeps_non_ascii_display_names_readable` covers From, To (Chinese and German names), a quoted Cc name with a comma, and Bcc. `test_partition_email_accepts_non_ascii_addresses` covers an RFC 6532 From and To. Both fail on `main` (the encoded-word, and the `UnicodeEncodeError`) and pass here. - `test_unstructured/partition/test_email.py`: 75 passed, 1 skipped. `test_msg.py` gives the same result as on `main`: one test fails locally on both because the image and PDF extras aren't installed. - `ruff check` and `ruff format --check` pass on the changed files. - CHANGELOG entry under 0.27.27, with the version bumped. <!-- This is an auto-generated description by cubic. --> <a href="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4543?utm_source=github" rel="nofollow">https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4543?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-dark.svg%2522%253E%253Csource" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-light.svg%2522%253E%253Cimg" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="View guided diff" src="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
fix(xlsx): read .xls files whose Workbook stream ends in padding (#4545) ## What `partition_xlsx()` now reads `.xls` files whose `Workbook` stream ends in a few bytes of padding, such as many files written by `xlwt`. Before, these raised `struct.error: unpack requires a buffer of 4 bytes` from the msoffcrypto encryption check. Closes #4544. ``` xlwt workbook, 3 rows (example-docs/xlwt-padded-workbook.xls) Before: struct.error: unpack requires a buffer of 4 bytes After: [Table("Fruit Count Apple 3 Pear 7")] ``` ## Why msoffcrypto's `is_encrypted()` scans an XLS `Workbook` stream as 4-byte record headers, looking for a `FilePass` record. `xlwt` pads the stream with zeros to a 4096-byte boundary, and when fewer than 4 bytes of padding are left, the scan raises (nolze/msoffcrypto-tool#83). A `struct.error` from that check is now read as "not encrypted". This does not let a password-protected `.xls` through: its `FilePass` record directly follows the `BOF` record, so the scan finds it before any padding. msoffcrypto's `rc4cryptoapi_password.xls` and `xor_password_123456789012345.xls` samples are still rejected with "XLSX file is password protected.", and its `plain.xls` sample still partitions. ## Validation - `test_partition_xlsx_reads_xls_whose_workbook_stream_ends_in_padding` partitions a 5.6 KB `xlwt` fixture. It fails on `main` with the `struct.error` and passes here. - `test_unstructured/partition/test_xlsx.py`: 108 passed. - `ruff check` and `ruff format --check` pass on the changed files. - CHANGELOG entry under 0.27.27, with the version bumped. #4543 also bumps to 0.27.27; I'll rebase whichever merges second. <!-- This is an auto-generated description by cubic. --> <a href="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4545?utm_source=github" rel="nofollow">https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4545?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-dark.svg%2522%253E%253Csource" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-light.svg%2522%253E%253Cimg" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="View guided diff" src="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
fix(ocr): include underlying error when loading OCR agent fails (#4491) ### Summary Fixes #3276. When `OCRAgent.get_instance()` fails to import or load a requested OCR agent module (e.g. missing dependencies or an `AttributeError`), it previously raised a generic `RuntimeError` without the original exception details or exception chaining (`from e`): ```python raise RuntimeError( "Could not get the OCRAgent instance. Please check the OCR package and the " "OCR_AGENT environment variable." ) ``` As reported in #3276 and discussed with @scanny, this masks the underlying error (such as `ImportError: No module named '...'`), making troubleshooting difficult. ### Changes 1. In `unstructured/partition/utils/ocr_models/ocr_interface.py`, include `{e}` in the raised `RuntimeError` message and chain the original exception with `raise ... from e`. 2. In `test_unstructured/partition/utils/ocr_models/test_ocr_interface.py`, update `and_it_raises_when_the_requested_agent_cannot_be_loaded` to verify that the underlying exception message is present in the raised `RuntimeError` and that `__cause__` references the original exception. 3. Update `CHANGELOG.md` under `## 0.27.8-dev0` / `### Fixes`. ### Testing - Ran targeted unit tests: ```bash pytest test_unstructured/partition/utils/ocr_models/test_ocr_interface.py -v ``` (All 10 tests passed) - Ran regression test suites: ```bash pytest test_unstructured/cleaners/test_core.py test_unstructured/partition/test_text.py ``` (All 197 tests passed) - Linting and formatting checks: ```bash ruff check unstructured/partition/utils/ocr_models/ocr_interface.py test_unstructured/partition/utils/ocr_models/test_ocr_interface.py ruff format --check unstructured/partition/utils/ocr_models/ocr_interface.py test_unstructured/partition/utils/ocr_models/test_ocr_interface.py ``` <!-- This is an auto-generated description by cubic. --> <a href="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4491?utm_source=github" rel="nofollow">https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4491?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-dark.svg%2522%253E%253Csource" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-light.svg%2522%253E%253Cimg" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
fix: preserve MathML content in HTML partitioning (#4478) ## Summary This PR adds support for preserving inline MathML content in the v1 HTML parser used by `partition_html()`. Previously, `<math>` was not assigned a custom element class. As a result, it fell back to `DefaultElement`, whose contents are intentionally skipped. This could silently remove mathematical expressions from otherwise valid extracted text. For example, HTML such as: ```html <p> The error rate is <math> <semantics> <annotation encoding="application/x-tex"> 10^{-4} </annotation> </semantics> </math> per token. </p> ``` could be extracted as: ```text The error rate is per token. ``` This PR introduces a `Math` phrasing element that preserves a textual representation of the expression when one is available. ## Changes * Adds a `Math` class derived from `Phrasing`. * Registers `<math>` elements with the v1 HTML parser. * Searches for recognized TeX/LaTeX annotations within the MathML subtree. * Supports the following annotation encodings: * `application/x-tex` * `tex` * `latex` * `application/x-latex` * Falls back to the `<math>` element's `alttext` attribute when no supported TeX annotation is available. * Preserves text following the closing `</math>` element by emitting the element tail using the existing phrasing behavior. The parser prefers an explicit TeX representation because flattening presentation MathML with `itertext()` can lose mathematical structure. For example, a superscript expression such as `10^-4` could otherwise be flattened into ambiguous plain text such as `10-4`. ## Why The HTML parser's documented principle is that the browser rendering is the document. However, before this change, standards-based MathML content could be rendered correctly by a browser while being completely removed during HTML partitioning because `<math>` was treated as an unrecognized element. This is especially problematic because the failure is silent. Surrounding paragraph text remains intact, so consumers of `partition_html()` may receive syntactically valid but incomplete text. Treating `<math>` as a specialized phrasing element allows mathematical content to participate naturally in surrounding paragraph text without requiring the parser to understand every individual Presentation MathML element such as `<mrow>`, `<msup>`, `<mfrac>`, `<mi>`, or `<mn>`. ## Scope This PR intentionally does not attempt to implement a complete MathML-to-text converter. It currently preserves mathematical content when a reliable textual representation is directly available through: 1. a recognized TeX/LaTeX `<annotation>`, or 2. the `<math>` element's `alttext` attribute. Other representations, such as Content MathML, arbitrary `annotation-xml` formats, OpenMath, SVG, or unsupported custom annotation encodings, are outside the scope of this change. ## Example Input: ```html <p> Complexity decreases from <math alttext="O(n^2)"></math> to <math> <semantics> <annotation encoding="application/x-tex"> O(n^2/r) </annotation> </semantics> </math>. </p> ``` Before: ```text Complexity decreases from to . ``` After: ```text Complexity decreases from O(n^2) to O(n^2/r). ``` ## Testing Tests have been added for MathML behavior including: * extracting `application/x-tex` annotations, * supported TeX/LaTeX encoding variants, * falling back to `alttext`, * preferring a supported TeX annotation when both annotation and `alttext` are present, * preserving surrounding and tail text, * multiple inline `<math>` elements in one paragraph, * ignoring unsupported annotation encodings, * realistic nested Presentation MathML, * interaction with existing phrasing/emphasis behavior, * and regression coverage for ordinary phrasing elements. Reviewer verification can also be performed with: ```python from unstructured.partition.html import partition_html html = r""" <p> The loss difference is on the order of <math> <semantics> <annotation encoding="application/x-tex"> 10^{-4} </annotation> </semantics> </math>. </p> """ elements = partition_html(text=html) for element in elements: print(element.text) ``` Expected output: ```text The loss difference is on the order of 10^{-4}. ``` <!-- This is an auto-generated description by cubic. --> <a href="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4478?utm_source=github" rel="nofollow">https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4478?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-dark.svg%2522%253E%253Csource" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-light.svg%2522%253E%253Cimg" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
feat(image): partition WEBP images like PNG and JPEG (#4537) ## What `.webp` images now partition like PNG and JPEG. ``` partition("photo.webp") Before: UnsupportedFileFormatError: Partitioning is not supported for the FileType.UNK file type. After: [Title(...), NarrativeText(...), ...] # each with metadata.filetype == "image/webp" ``` ## Why `FileType` had no WEBP member, so detection fell through to `UNK` and `partition()` rejected every `.webp`. Pillow already decodes WEBP and `partition_image()` already accepts it, so the change is one `FileType.WEBP` member, which detection by content type, by libmagic or `filetype`, and by extension all read. `webp` also joins the image types that skip table inference by default, with PNG, JPEG and HEIC, and the types runtime telemetry counts as images. Two behaviours change beyond files named `.webp`: - `partition_email()` and `partition_msg()` used to skip a WEBP attachment with a warning. With the `image` extra installed they now partition it; without the extra they still skip it with a warning. A WEBP attachment that fails image processing, for example because Tesseract is missing, now raises and fails the whole call, as a PNG or JPEG attachment already does. - A WEBP saved under another image extension is detected as `image/webp` from its content, where its extension decided before. `IMAGE_URL_PATTERN` stays as it is. Its only user is `extract_image_urls_from_html()`, a cleaner that partitioning never calls, so it doesn't decide whether a `.webp` partitions. Adding `webp` to it is a separate change. ## Validation - `detect_filetype()`: WEBP joins each table HEIC is in, reading the new `example-docs/img/DA-1p.webp` (a WEBP copy of `DA-1p.jpg`). A WEBP copied to `renamed.png` is detected as WEBP, where `main` says PNG. - `test_auto_partition_webp_from_filename` partitions a lossless WEBP copy of `layout-parser-paper-fast.jpg` through `partition()` under `auto` and `ocr_only`, as the BMP test does with its own copy. - `test_auto_partition_webp_skips_table_inference_by_default_like_png` checks the `infer_table_structure` value `partition()` passes to the image partitioner: true for BMP, false for HEIC, JPEG, PNG and WEBP. Dropping `"webp"` from the default fails it. - Two `partition_email()` tests attach `DA-1p.webp` to a message built with `EmailMessage`. When the image dependency check fails, the attachment is skipped with an `ImportError` warning from `partition_image()`. With the image partitioner stubbed, the attachment's elements carry `filetype` `image/webp`, `filename` `DA-1p.webp` and `attached_to_filename`. `partition_msg()` follows the same attachment rule but has no WEBP test of its own. - The telemetry strategy test gains a `webp` row. Against `main`'s source every new WEBP case fails. The table test's BMP, HEIC, JPEG and PNG rows pass there, since they pin the existing defaults. CI ran all 19 new items and passed them. On Python 3.11, 3.12 and 3.13, `test_unit` reports 4209 passed and 15 skipped, 19 items more than `main`'s push run of the same base (4195 passed, 10 skipped). Pull-request runs skip five tests that push runs execute. The docs' open-source supported-file-types and partitioning-strategies pages list `.heic` but not `.webp`; that update belongs in the docs repository. <!-- This is an auto-generated description by cubic. --> <a href="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4537?utm_source=github" rel="nofollow">https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4537?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-dark.svg%2522%253E%253Csource" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="/?u=https%3A%2F%2Fgithub.com%2F%253Ca href="/?u=https%3A%2F%2Fwww.cubic.dev%2Fbuttons%2Freview-in-cubic-light.svg%2522%253E%253Cimg" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="View guided diff" src="/?u=https%3A%2F%2Fgithub.com%2F%253Ca%2520href%3D"https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a" rel="nofollow">https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
PreviousNext