Big Data Archive

How Machine Learning Can Transform Content Management- Part III

In a previous post, I highlighted how machine learning can be applied to content management. In that article, I described how content analysis could identify social, emotional and linguistic tone of an article. This informs a content author further on his writing style, and helps the author understand which style resonates most with his readers.

A follow up article outlined the use of Amazon’s Artificial Intelligence Services to perform text to speech translations.

At the most recent AWS re:Invent conference, Amazon announced Comprehend, a natural language processing (NLP) service that uses machine learning to find insights and relationships in text. This seemed like a good occasion to revisit the initial article, and explore how Comprehend can provide content intelligence.

Amazon Comprehend

Amazon Comprehend has two sets of services, one targeted for interactive use with responses available in hundreds of milliseconds, the other more batch oriented.

The interactive services has four core API’s:

  • Entity returns the named entities (“Person”, “Organization”, “Locations”, etc.) within the text being analyzed.
  • Key Phrases returns key phrases and a confidence score to support that this is a key phrase.
  • Sentiment returns the overall sentiment of a document (Positive, Negative, Neutral, or Mixed).
  • Language returns the dominant language in the text and a confidence score to support that a language is dominant.

The batch-oriented services can scan a set of documents, and return a summary of key topics in these documents.

Integrating Comprehend with Adobe AEM Content Fragments

Similar to the original article, we integrated AWS Comprehend with AEM Content Fragments.

The same piece of content was leveraged. Unfortunately, AWS Comprehend only supports up to 5000 characters, so a subset of the overall text was selected.

Entities, sentiment and language were assigned as metadata tags to the document. Only entities with more than two occurrences were included. Entities provide a categorization (such as date, location, event, organization, person, commercial_item, etc), and the category is included in the tag structure.

In this example, sentiment is also included. Overall, it would make sense to only include tone if a certain confidence threshold has been met.

The key phrase response was used to generate a highlighted version of the content fragment. Similar to sentiment and entities, each phrase is scored with a level of confidence, and this can be used to fine tune phrase detection.

If you want to explore the code, it is posted on Github as part of the https://github.com/razorfish/contentintelligence repository.

Conclusion

In the original article, I listed a couple of potential reasons why leveraging machine learning in the content management process could be relevant:

  • How does content come across to readers? Which tones resonate the most? What writer is successful with which tone?
  • What topics are covered most frequently? How much duplication exists?
  • What is the article about?
  • What are the key topics covered in this article?

The Amazon Comprehend solution can help with some of these as shown in this proof of concept.

For topic classification, the approach taken here with entities can be valuable. Especially the ability to provide categorizations of entities can help with providing more easily readable insights.

Key phrase detection can help with quickly reviewing what an article is about, identify repetition and can especially be helpful in content management scenarios with high volume content production.

AUTHOR
Martin Jacobs
GVP, Technology

How Machine Learning Can Transform Content Management- Part II

In a previous post, I highlighted how machine learning can be applied to content management. In that article, I described how content analysis could identify social, emotional and language tone of an article. This informs a content author further on his writing style, and helps the author understand which style resonates most with his readers.

At the last AWS re:Invent conference, Amazon announced a new set of Artificial Intelligence Services, including natural language understanding (NLU), automatic speech recognition (ASR), visual search, image recognition and text-to-speech (TTS).

In previous posts, I discussed how the Amazon visual search and image recognition services could benefit digital asset management. This post will highlight the use of some of the other services in a content management scenario.

Text to Speech

From a content perspective, text to speech is a potential interesting addition to a content management system. One reason could be to assist visitors with vision problems. Screen readers can already perform many of these tasks, but leveraging the text to speech service provides publishers more control over the quality and tone of the speech.

Another use case is around story telling. One of the current trends is to create rich immersive stories, with extensive photography and visuals. This article from the New York Times describing the climb of El Capitan in Yosemite is a good example. With the new text to speech functionality, the article can easily be transformed into a self playing presentation with voice without any additional manual efforts, such as recruiting voice talent and the recording process.

Amazon Polly

Amazon Polly (the Amazon AI text to speech service) turns text into lifelike speech. It uses advanced deep learning technologies to synthesize speech that sounds like a human voice. Polly includes 47 lifelike voices that spread across 24 languages. Polly is service that can be used in real-time scenarios. It can used to retrieve a standard audio file (such as MP3) that can be stored and used at a later point. The lack of restrictions on storage and reuse of voice output makes it a great option for use in a content management system.

Polly and Adobe AEM Content Fragments

In our proof of concept, the Amazon AI text to speech service was applied to Content Fragments in Adobe AEM.

Content Fragment

As shown above, Content Fragments allow you to create channel-neutral content with (possibly channel-specific) variations. You can then use these fragments when creating pages. When creating a page, the content fragment can be broken up into separate paragraphs, and additional assets can be added at each paragraph break.

After creating a Content Fragment, the solution passes the content to Amazon Polly for retrieving the audio fragments with spoken content. It will generate a few files, one for the complete content fragment (main.mp3), and a set of files broken up by paragraph (main_1.mp3, main_2.mp3, main_3.mp3). The audio files are stored as associated content for the master Content Fragment.

Results

When authoring a page that uses this Content Fragment, the audio fragments are visible in the sidebar, and can be added to the page if needed. With this capability in place, developing a custom story telling AEM component to support scenarios like the New York Times article becomes relatively simple.

If you want to explore the code, it is posted on Github as part of the https://github.com/razorfish/contentintelligence repository.

Conclusion

This post highlighted how an old problem can now be addressed in a new way. The text to speech service will make creating more immersive experiences easier and more cost effective. Amazon Polly’s support for 24 languages opens up new possibilities as well. Besides the 2 examples mentioned in this post, it could also support scenarios such as interactive kiosks, museum tour guides and other IOT, multichannel or mobile experiences.

With these turnkey machine-learning services in place, creative and innovative thinking is critical to successfully solve challenges in new ways.

AUTHOR
Martin Jacobs
GVP, Technology

How Machine Learning Can Transform Content Management

In previous posts, I explored the opportunities for machine learning in digital asset management, and, as a proof-of-concept, integrated a DAM solution (Adobe AEM DAM) with a set of machine learning APIs.

But the scope of machine learning extends much further. Machine learning can also have a profoundly positive impact on content management.

In this context, machine learning is usually associated with content delivery. The technology can be used to deliver personalized or targeted content to website visitors and other content consumers. Although this is important, I believe there is another opportunity that stems from incorporating machine learning into the content creation process.

Questions Machine Learning Can Answer

During the content creation process, content can be analyzed by machine learning algorithms to help address some key questions:

  • How does content come across to readers? Which tones resonate the most? What writer is successful with which tone? Tone analysis can help answer that.
  • What topics are covered most frequently? How much duplication exists? Text clustering can help you analyze your overall content repository.
  • What is the article about? Summarization can extract relevant points and topics from a piece of text, potentially helping you create headlines.
  • What are the key topics covered in this article? You can use automatic topic extraction and text classification to create metadata for the article to make it more linkable and findable (both within the content management tool internally and via search engines externally).

Now that we know how machine learning can transform the content creation process, let’s take a look at a specific example of the technology in action.

An Implementation with AEM 6.2 Content Fragments and IBM Watson

Adobe just released a new version of Experience Manager, which I discussed in a previous post. One of the more important features of AEM 6.2 is the concept of “Content Fragments.”

In the past, content was often tied to pages. But Content Fragments allow you to create channel-neutral content with (possibly channel-specific) variations. You can then use these fragments when creating pages.

Content Fragments are treated as assets, which makes them great candidates for applying analysis and classification. Using machine learning, we’re able to analyze the tone of each piece of content. Tones can then be associated with specific pieces of content.

In the implementation, we used IBM Bluemix APIs to perform tone analysis. The Tone Analyzer computes emotional tone (joy, fear, sadness, disgust or anger), social tone (openness, conscientiousness, extraversion, agreeableness or emotional range) and language tone (analytical, confident or tentative).

The Tone Analyzer also provides insight on how the content is coming across to readers. For each sub-tone, it provides a score between 0 and 1. In our implementation, we associated a sub-tone with the metadata only if the score was 0.75 or higher.

The Results of Our Implementation

If you want to take a look, you’ll find the source code and setup instructions for the integration of Content Fragments with IBM Watson Bluemix over on GitHub.

We ran our implementation against the text of Steve Jobs’ 2005 Stanford commencement speech, and the results are shown below. For every sub-tone with a score of 0.75 or higher, a metadata tag was added.

Results

The Takeaway from Our Implementation

Machine learning provides a lot of opportunities during the content creation process. But it also requires authoring and editing tools that seamlessly integrate with these capabilities to really drive adoption and make the use of those machine-learning insights a common practice.

Personally, I can’t wait to analyze my own posts from the last few months using this technology. That analysis, combined with LinkedIn analytics, will enable me to explore how I can improve my own writing and make it more effective.

Isn’t that what every content creator wants?

AUTHOR
Martin Jacobs
GVP, Technology