Registry / utility / tactful-tokenizer

tactful-tokenizer

JSON →
library0.0.5rbrubygemsunverified

Uses a naive Bayesian model trained on the Brown and WSJ corpora to provide high quality sentence tokenization.

utility
gem install tactful_tokenizer
github ↗homepage ↗docs ↗
INSTALL
IMPORT
SIG · TACTFUL-TOKENIZER
T
tactful-tokenizer
utilityrubyv0.0.5
harness data pending
Install & Compatibility
Where this runs

No compatibility data collected yet for this library.

Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

TactfulTokenizer
✓ require 'tactful_tokenizer'

Tokenize a string into sentences.

require 'tactful_tokenizer' tokenizer = TactfulTokenizer::Model.new sentences = tokenizer.tokenize('Hello world. This is a test.') puts sentences.inspect
Debug
Known issues

No known issues recorded.

Upgrade
Version history
0.0.5latest on RubyGems
Audit
Dependencies

No dependency data recorded yet.

Agent activity
6 hits · last 30 days
node
6
Resources
githubhttps://github.com/zencephalon/Tactful_Tokenizer ↗
homepagehttp://github.com/zencephalon/Tactful_Tokenizer ↗
docshttp://www.rubydoc.info/gems/tactful_tokenizer/ ↗
tactful-tokenizer — gem install tactful-tokenizer · libregistry