Home > Uncategorized > Tagging Content With IBM Watson

Tagging Content With IBM Watson

I recently had a chance to work on an interesting 3rd party integration for a client. We used IBM Watson to automatically categorize and tag the clients content.

The clients Editors had not been classifying content consistently on their site. They were looking for an automated way to clean up their content without manually reviewing everything. Watson gave us the ability to standardize content into IAB Categories, and fill in the gaps in their existing tags.

In theory, this sounds like a great idea. But in practice, Watson can yield mixed results.

Natural Language Understanding

Let’s put first things first. IBM Watson is a Cloud-based offering that includes a whole host of services and APIs. From Language translation, Tone analysis, Speech to Text all are available under the Watson umbrella. The API to use for classifying text is called Natural Language Understanding or NLU.

The NLU API uses REST conventions. It takes classification options in the POST body and returns JSON results. The things that the API can identify in your text are called features. The features that were most relevant to us were Categories & Keywords.

Pricing

The NLU API is charged based on Units of Usage — Data Units & Enrichment Units. 10k characters equal 1 NLU unit & each feature equals 1 NLU unit as well.

Eg:- Classifying 5 posts with 1 feature is equivalent to 1 post with 5 features.

Tiers
Pricing
1 – 250,000
$0.003 USD/NLU Item
250,001 – 5,000,000
$0.001 USD/NLU Item
5,000,000+
$0.0002 USD/NLU Item

 

This roughly translates to around 20k NLU units to classify 5000 articles. And will typically be a one-time cost of < $200 for most medium to large sites.

A lite/trial tier is also offered by IBM. It gives us 30k NLUs free every month. The caveat is that the services created in the account are deleted after 30 days.

Improving Watson Results

The NLU API can classify both HTML & Text content. However, I found that manually stripping tags and sending plain text to Watson almost always yielded better results.

Shortcodes inside content can sometimes contain valuable text. For instance, I found the caption shortcode often contained keywords that would significantly improve the relevance of NLU results.

Common phrases were another cause of poor results. Editors on client’s often end an article with a phrase like “This article was featured in the clients publication on a Date”. Sending this to NLU will often give back the Arts & Magazine as Category matches. Removing these common phrases also helped improved relevancy.

The Title of an article is another good way to improve relevance. Inserting it at the top of the content before sending to the NLU API yielded good results even in cases when the content was quite brief.

Lastly, we excluded Fictional content from IBM Watson. Manual classification is still needed in places like this where Watson results can’t be applied.

For the client we initially intended to use Watson during migration of old content. But now we are also integrating it into the WP-Admin to help Editors classify content as it is being created.

Leave a Reply

Your email address will not be published. Required fields are marked *