# NLTK **Supported Labeling Types**: `Span labeling` NLTK (Natural Language Toolkit) is an open-source Python library for natural language processing (NLP). It provides tools for text preprocessing such as tokenization, stemming, lemmatization, part-of-speech tagging, and more. In the context of our labeling platform, NLTK can be integrated to support various preprocessing tasks that help improve label consistency and model training quality. Its ease of use and rich set of linguistic resources make it a useful option for preparing and analyzing text data before or during the labeling process.
Image of ML Assisted with NLTK

ML Assisted with NLTK

### **Model Details** * NLTK POS-tagging is performed using `nltk.pos_tag`, which internally utilizes the `nltk.PerceptronTagger`. This is a fast and accurate approach for part-of-speech tagging in English. * The underlying models are trained on the Wall Street Journal section of the Penn Treebank, providing strong performance on formal, edited text common in business documents. * The tagger assigns grammatical categories to words based on the **UPenn Treebank Tagset**, which includes categories like nouns, verbs, adjectives, adverbs, and more. * Fully integrated into the Datasaur Intelligence container for consistent, dependency-free operation. > Greedy Averaged Perceptron tagger, as implemented by Matthew Honnibal. See more implementation details here: > ### Usage * Text preprocessing for consistency and model training improvements. * Supports syntactic analysis in annotation workflows. * Tag set: [UPenn Treebank Tag Set](https://docs.datasaur.ai/assisted-labeling/ml-assisted-labeling/nltk#upenn-treebank-tagset) * The detailed available [here](https://docs.datasaur.ai/assisted-labeling/ml-assisted-labeling/nltk#appendix) ### References * * * ## Appendix ### NLTK Treebank
$dollar e.g. $, -$, --$, A$, C$, HK$, M$, NZ$, S$, U.S.$, US$
''closing quotation mark e.g. ', ''
(opening parenthesis e.g. (, [, {
,comma e.g. ,
--dash e.g. --
.sentence terminator e.g. ., !, ?
:colon or ellipsis e.g. :, ;, ...
``opening quotation mark e.g. `, ``
### Treebank Tagset
TagDescription
CCconjunction, coordinating e.g. &, 'n, and, both, but, either, et, for, less, minus, neither, nor, or, plus, so, therefore, times, v., versus, vs., whether, yet
CDnumeral, cardinal e.g. mid-1890, nine-thirty, forty-two, one-tenth, ten, million, 0.5, one, forty-, seven, 1987, twenty, '79, zero, two, 78-degrees, eighty-four, IX, '60s, .025, fifteen, 271,124, dozen, quintillion, DM2,000, ...
DTdeterminer e.g. all, an, another, any, both, del, each, either, every, half, la, many, much, nary, neither, no, some, such, that, the, them, these, this, those
EXexistential there e.g. there
FWforeign word e.g. gemeinschaft, hund, ich, jeux, habeas, Haementeria, Herr, K'ang-si, vous, lutihaw, alai, je, jour, objets, salutaris, fille, quibusdam, pas, trop, Monte, terram, fiche, oui, corporis, ...
INpreposition or conjunction, subordinating e.g. astride, among, uppon, whether, out, inside, pro, despite, on, by, throughout, below, within, for, towards, near, behind, atop, around, if, like, until, below, next, into, if, beside, ...
JJadjective or numeral, ordinal e.g. third, ill-mannered, pre-war, regrettable, oiled, calamitous, first, separable, ectoplasmic, battery-powered, participatory, fourth, still-to-be-named, multilingual, multi-disciplinary, ...
JJRadjective, comparative e.g. bleaker, braver, breezier, briefer, brighter, brisker, broader, bumper, busier, calmer, cheaper, choosier, cleaner, clearer, closer, colder, commoner, costlier, cozier, creamier, crunchier, cuter, ...
JJSadjective, superlative e.g. calmest, cheapest, choicest, classiest, cleanest, clearest, closest, commonest, corniest, costliest, crassest, creepiest, crudest, cutest, darkest, deadliest, dearest, deepest, densest, dinkiest, ...
LSlist item marker e.g. A, A., B, B., C, C., D, E, F, First, G, H, I, J, K, One, SP-44001, SP-44002, SP-44005, SP-44007, Second, Third, Three, Two, *, a, b, c, d, first, five, four, one, six, three, two
MDmodal auxiliary e.g. can, cannot, could, couldn't, dare, may, might, must, need, ought, shall, should, shouldn't, will, would
NNnoun, common, singular or mass e.g. common-carrier, cabbage, knuckle-duster, Casino, afghan, shed, thermostat, investment, slide, humour, falloff, slick, wind, hyena, override, subhumanity, machinist, ...
NNPnoun, proper, singular e.g. Motown, Venneboerger, Czestochwa, Ranzer, Conchita, Trumplane, Christos, Oceanside, Escobar, Kreisler, Sawyer, Cougar, Yvette, Ervin, ODI, Darryl, CTCA, Shannon, A.K.C., Meltex, Liverpool, ...
NNPSnoun, proper, plural e.g. Americans, Americas, Amharas, Amityvilles, Amusements, Anarcho-Syndicalists, Andalusians, Andes, Andruses, Angels, Animals, Anthony, Antilles, Antiques, Apache, Apaches, Apocrypha, ...
NNSnoun, common, plural e.g. undergraduates, scotches, bric-a-brac, products, bodyguards, facets, coasts, divestitures, storehouses, designs, clubs, fragrances, averages, subjectivists, apprehensions, muses, factory-jobs, ...
PDTpre-determiner e.g. all, both, half, many, quite, such, sure, this
POSgenitive marker e.g. ', 's
PRPpronoun, personal e.g. hers, herself, him, himself, hisself, it, itself, me, myself, one, oneself, ours, ourselves, ownself, self, she, thee, theirs, them, themselves, they, thou, thy, us
PRP$pronoun, possessive e.g. her, his, mine, my, our, ours, their, thy, your
RBadverb e.g. occasionally, unabatingly, maddeningly, adventurously, professedly, stirringly, prominently, technologically, magisterially, predominately, swiftly, fiscally, pitilessly, ...
RBRadverb, comparative e.g. further, gloomier, grander, graver, greater, grimmer, harder, harsher, healthier, heavier, higher, however, larger, later, leaner, lengthier, less-, perfectly, lesser, lonelier, longer, louder, lower, more, ...
RBSadverb, superlative e.g. best, biggest, bluntest, earliest, farthest, first, furthest, hardest, heartiest, highest, largest, least, less, most, nearest, second, tightest, worst
RBSadverb, superlative e.g. best, biggest, bluntest, earliest, farthest, first, furthest, hardest, heartiest, highest, largest, least, less, most, nearest, second, tightest, worst
RBSparticle e.g. aboard, about, across, along, apart, around, aside, at, away, back, before, behind, by, crop, down, ever, fast, for, forth, from, go, high, i.e., in, into, just, later, low, more, off, on, open, out, over, per, pie, raising, start, teeth, that, through, under, unto, up, up-pp, upon, whole, with, you
RPparticle e.g. aboard, about, across, along, apart, around, aside, at, away, back, before, behind, by, crop, down, ever, fast, for, forth, from, go, high, i.e., in, into, just, later, low, more, off, on, open, out, over, per, pie, raising, start, teeth, that, through, under, unto, up, up-pp, upon, whole, with, you
SYMsymbol e.g. %, &, ', '', ''., ), )., *, +, ,., <, =, >, @, A[fj], U.S, U.S.S.R, *, **, ***
TO"to" as preposition or infinitive marker e.g. to
UHinterjection e.g. Goodbye, Goody, Gosh, Wow, Jeepers, Jee-sus, Hubba, Hey, Kee-reist, Oops, amen, huh, howdy, uh, dammit, whammo, shucks, heck, anyways, whodunnit, honey, golly, man, baby, diddle, hush, sonuvabitch, ...
VBverb, base form e.g. ask, assemble, assess, assign, assume, atone, attention, avoid, bake, balkanize, bank, begin, behold, believe, bend, benefit, bevel, beware, bless, boil, bomb, boost, brace, break, bring, broil, brush, build, ...
VBDverb, base form e.g. ask, assemble, assess, assign, assume, atone, attention, avoid, bake, balkanize, bank, begin, behold, believe, bend, benefit, bevel, beware, bless, boil, bomb, boost, brace, break, bring, broil, brush, build, ...
VBGverb, present participle or gerund e.g. telegraphing, stirring, focusing, angering, judging, stalling, lactating, hankerin', alleging, veering, capping, approaching, traveling, besieging, encrypting, interrupting, erasing, wincing, ...
VBNverb, past participle e.g. multihulled, dilapidated, aerosolized, chaired, languished, panelized, used, experimented, flourished, imitated, reunifed, factored, condensed, sheared, unsettled, primed, dubbed, desired, ...
VBPverb, present tense, not 3rd person singular e.g. predominate, wrap, resort, sue, twist, spill, cure, lengthen, brush, terminate, appear, tend, stray, glisten, obtain, comprise, detest, tease, attract, emphasize, mold, postpone, sever, return, wag, ...
VBZverb, present tense, 3rd person singular e.g. bases, reconstructs, marks, mixes, displeases, seals, carps, weaves, snatches, slumps, stretches, authorizes, smolders, pictures, emerges, stockpiles, seduces, fizzes, uses, bolsters, slaps, speaks, pleads, ...
WDTWH-determiner e.g. that, what, whatever, which, whichever
WPWH-determiner e.g. that, what, whatever, which, whichever
WP$WH-pronoun, possessive e.g. whose
WRBWH-adverb e.g. how, however, whence, whenever, where, whereby, whereever, wherein, whereof, why
### References * UPenn Treebank Docs * `python -c "import nltk; nltk.help.upenn_tagset()"` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.datasaur.ai/assisted-labeling/ml-assisted-labeling/nltk.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.