Vulnerability GHSA-qg35-44qx-7vrr

Summary

YCOECorpusReader builds two nested corpus sub-readers whose roots are derived from self.root.join("psd") and self.root.join("pos") (nltk/corpus/reader/ycoe.py). Because FileSystemPathPointer.join() (nltk/data.py) performs a purely lexical containment check and never resolves symlinks, and because CorpusReader.__init__ (nltk/corpus/reader/api.py) validates a sub-reader root only against the global NLTK data sandbox and not against the enclosing corpus root, a directory symlink named psd or pos planted inside a YCOE corpus lets the reader anchor its nested sub-readers outside the YCOE corpus root. An attacker who controls the contents of a YCOE corpus loaded under pathsec.ENFORCE=True can make parsed_sents(), tagged_words(), and related methods return file content from other locations inside the allowed data sandbox, defeating the per-corpus-root containment boundary the pathsec controls are meant to enforce.

Root Cause

NLTK's pathsec controls are designed to keep each corpus reader confined to its own corpus root. The scoped containment is enforced through validate_path(..., required_root=self._root) at read time. However, the containment invariant is violated at reader-construction time for nested readers:

• FileSystemPathPointer.join() accepts a component that is a directory symlink because it only checks the joined string lexically (os.path.normpath plus a startswith prefix test) and never calls resolve().
• CorpusReader.__init__ validates the resulting sub-reader root string, but calls validate_path without required_root, so only the global-sandbox layer runs; the scoped per-corpus-root layer is skipped. The symlink is resolved by that global check, its target is found to be inside the allowed sandbox, and the sub-reader is accepted with its root pointing at the escaped target.

From that point every read the sub-reader performs is scoped to the already-escaped root, so no later guard re-anchors it to the YCOE corpus root.

Affected functions:
- nltk.corpus.reader.ycoe.YCOECorpusReader.__init__   (nltk/corpus/reader/ycoe.py) builds nested sub-readers from self.root.join("psd")/("pos") without containing them to the parent root
- nltk.corpus.reader.api.CorpusReader.__init__        (nltk/corpus/reader/api.py) validates a sub-reader root only against the global sandbox (validate_path without required_root)
- nltk.data.FileSystemPathPointer.join               (nltk/data.py) applies a lexical prefix check only and never resolves symlinks

Impact

An attacker who controls the directory contents of a YCOE corpus that is loaded by a victim application under pathsec.ENFORCE=True (the default in the affected releases) can read file content that lives outside the YCOE corpus root but inside an allowed NLTK data root. The disclosure is bounded to the global NLTK data sandbox: a symlink whose target resolves outside every allowed root is rejected at construction, so this is not an arbitrary-filesystem read. The demonstrated capability is read-only cross-corpus-root disclosure. In a multi-tenant or automated-ingestion deployment the accessible scope includes other corpora placed under the shared data path and the shared system temporary directory, which is a default allowed root and can hold other processes' sensitive temporary files. There is no integrity or availability impact.

Proof of Concept

Dependencies: a released nltk (3.10.0 through 3.10.3) on a POSIX host. No third-party packages. The only allowed data root is a fresh temp sandbox; ENFORCE is left at its default of True.

Steps:

1. Create a sandbox directory and register it as the sole nltk.data.path entry.
2. Inside the sandbox create a "victim" directory holding doc1.psd / doc1.pos outside the YCOE corpus root.
3. Create a YCOE corpus root whose psd and pos entries are directory symlinks pointing at the victim directory.
4. Construct YCOECorpusReader on that root and read it.

import os, tempfile, nltk
from nltk import pathsec
sandbox = os.path.realpath(tempfile.mkdtemp())
pathsec.ENFORCE = True
nltk.data.path = [sandbox]
pathsec._ALLOWED_ROOTS_CACHE = None
from nltk.corpus.reader.ycoe import YCOECorpusReader

victim = os.path.join(sandbox, "victim", "psd_secret")
victim_pos = os.path.join(sandbox, "victim", "pos_secret")
os.makedirs(victim); os.makedirs(victim_pos)
open(os.path.join(victim, "doc1.psd"), "w").write("(S (SECRET leaked-out-of-ycoe-root))\n")
open(os.path.join(victim_pos, "doc1.pos"), "w").write("SECRET_NN leaked-out-of-ycoe-root_NN\n")

ycoe = os.path.join(sandbox, "corpora", "ycoe_evil"); os.makedirs(ycoe)
os.symlink(victim, os.path.join(ycoe, "psd"))
os.symlink(victim_pos, os.path.join(ycoe, "pos"))

reader = YCOECorpusReader(ycoe)
print(reader.parsed_sents())
print(reader.tagged_words())

psd sub-reader real root : /tmp/ycoe_poc_zc35zihw/victim/psd_secret
ycoe corpus root         : /tmp/ycoe_poc_zc35zihw/corpora/ycoe_evil
parsed_sents()           : [Tree('S', [Tree('SECRET', ['leaked-out-of-ycoe-root'])])]
tagged_words()           : [('SECRET', 'NN'), ('leaked-out-of-ycoe-root', 'NN')]
ESCAPE CONFIRMED: YCOE read out-of-root content via psd/pos symlink

A negative control with real (non-symlink) psd/pos directories stays contained, and a symlink whose target is /etc is rejected at construction with a PermissionError, confirming the escape is bounded to the sandbox.

Attack Chain

1. Exposure: A victim application loads a YCOE corpus whose directory contents are attacker-controlled (a downloaded, shared, or planted corpus placed under an allowlisted NLTK data root), running with pathsec.ENFORCE=True.
2. Control: The attacker sets the corpus root's psd and pos entries to directory symlinks pointing at another directory inside the allowed sandbox.
3. Path: YCOECorpusReader.__init__ calls self.root.join("psd"); FileSystemPathPointer.join() returns a pointer to <root>/psd after a lexical-only check that never resolves the symlink.
4. Guard: CorpusReader.__init__ calls validate_path(root) without required_root, so the scoped per-corpus-root layer is skipped; only the global-sandbox layer runs and it resolves the symlink to an in-sandbox target and accepts it.
5. Primitive: The nested sub-reader's root is now the escaped target; find_corpus_fileids and the read-time validate_path(required_root=self._root) both anchor to that escaped root, so all reads are treated as in-root.
6. Result: parsed_sents(), tagged_words(), tagged_sents(), and words() return file content from outside the YCOE corpus root but inside the allowed sandbox: read-only cross-corpus-root disclosure.

Bypass Evidence

The relevant guards are the pathsec containment checks that were added to confine corpus readers to their roots. They do not block this path:

• FileSystemPathPointer.join() (nltk/data.py) checks joined == root or joined.startswith(root + os.sep) on the lexical, non-resolved path. A psd component that is a directory symlink yields <root>/psd, which satisfies the prefix test, and the symlink is never resolved. FileSystemPathPointer.__init__ uses os.path.abspath, which also does not resolve symlinks.
• CorpusReader.__init__ (nltk/corpus/reader/api.py) invokes validate_path(root, context="CorpusReader.__init__") with no required_root. In validate_path, the scoped layer is guarded by if required_root: and is therefore skipped; only the global-sandbox layer runs, which resolves the symlink target and accepts it because it lies inside nltk.data.path.
• The read-time guard validate_path(path, required_root=self._root) uses the sub-reader's own self._root, which is already the escaped target, so it cannot detect the escape. find_corpus_fileids computes resolved_root = os.path.realpath(root.path) on the escaped root, so its commonpath descent guard also confines to the escaped target rather than the YCOE root.

Refutation attempts that failed: a boundary test with a /etc symlink target is rejected at construction (PermissionError), so this is not arbitrary-filesystem read; a negative control with real psd/pos directories stays contained; and the escape reproduces against the exact published PyPI sdists for 3.10.0 and 3.10.3, so it is not an artifact of an unreleased development tree. No published advisory or fix addresses this reader: the currently released readers hardened for symlink escapes do not include YCOE's nested-reader root construction, and no fix commit touches nltk/corpus/reader/ycoe.py.

Affected Versions

• Ecosystem: PyPI
• Package: nltk
• Confirmed affected range: >= 3.10.0, <= 3.10.3
• Latest release checked: 3.10.3 (PyPI sdist sha256 bb9327a461c3811c2fa4900e03840401f2126adfb30c0072827c433bd2444ea4, uploaded 2026-08-12)
• Fix status: 3.10.4

The pathsec corpus-root containment control is active by default (ENFORCE=True) in the 3.10.x line, and the vulnerable YCOE nested-reader construction is present unchanged in every 3.10.x release. The escape was reproduced against the published 3.10.0 and 3.10.3 sdists. Version 3.9.4 ships with ENFORCE=False by default, so the per-corpus-root control is not active there by default; that un-hardened default is tracked separately and is not the boundary this report concerns. No later fixed release exists at the time of review.

Suggested Fix

Enforce the per-corpus-root containment invariant when a reader constructs a nested sub-reader. Two complementary options:

• In YCOECorpusReader.__init__, validate each nested root against the parent corpus root, for example validate_path(self.root.join("psd"), required_root=self.root) (and likewise for pos), so a sub-reader root that resolves outside the enclosing YCOE root is rejected.
• Harden FileSystemPathPointer.join() to reject a component whose resolved target escapes the pointer's own resolved root, rather than relying on a lexical prefix test, so no reader can be pointed outside its root by a symlinked component.

Apply the same review to any other reader that constructs nested sub-readers from a root-derived path. As an interim mitigation, only load YCOE corpora from trusted, attacker-inaccessible directories, and avoid placing untrusted corpora under a shared NLTK data root in multi-tenant or automated-ingestion pipelines.

Reported by zx (GitHub: @manus-pi).

Fix status (2026-09-28)

Status: IN PROGRESS (fix on an open pull request, not yet on develop).

PR: #3888 "Validate chat80.sql_query dbname for path containment (CWE-59/73)", open; head e0e890f51 at the time of this note.

Fix commits: 68434c042 "Enforce corpus-root containment in YCOE reader and StreamBackedCorpusView" and 2c563b13f (scopes the YCOE guard).

No probe. Regression test on the PR: nltk/test/unit/test_ycoe_streamview_pathsec.py (13 passed on the PR head e0e890f51). The advisory PoC re-run (psd and pos directory symlinks pointing outside the corpus root but inside the data sandbox): on the PR head YCOECorpusReader refuses with ValueError (Security Violation [YCOECorpusReader]: path escapes root); on develop 2fa3f48fe parsed_sents() and tagged_words() return the planted out-of-root content, so develop is still VULNERABLE.