Talkback is a web-based system to analyse data that has been specifically mined from social-media. The project has been built from a security-perspective to mine social-media for vulnerability references and then utilises the inventory of security-relevant users to identify trending items.
Talkback has been running for over a year, to read a blog post reflecting on the way the tool has slowly evolved and the data it's captured, click here.
SCMagazine Australia wrote a feature article on Talkback, which can be found here.
If you like this project, follow @volvent on Twitter to receive updates on Talkback and other upcoming public projects from Volvent Security.
One of the initial motivations behind mining Twitter for vulnerabilities was sparked by the fact public vulnerability data (e.g. NVD) can suffer from data-issues when information is missing or incomplete. Talkback can help compile a dynamic timeline for public vulnerabilities by leveraging discussions and references in social-media.
The second motivator is the difficulty of keeping up with information in a rapidly changing field. Social media as it's generally used does not make things easier, as the ratio of noise is too high. By utilising the user inventory built and maintained from the first motivator, it opens the possibility of capturing trending items within a specific community of users. The trending items interface demonstrates this and is a work in progress.
Here is a brief list of upcoming features to Talkback:
Talkback is a simple program that fits together (more or less) like so:
Weights are calculated by identifying parent items and comparing the similarity between them and potential child items. This method is used to workaround retweets, URL shorterners, hash-tags, and the rest of the junk that comes with Twitter.
The following table shows the value ranges and notes for each of the weight-classes:
| Weight | Value/Range | Timeline Gradient Colour | Special Notes |
|---|---|---|---|
| Fly | 0 | #ffce2e | - |
| Feather | 1:4 | #ff9d00 | - |
| Light | 5:9 | #ff7700 | - |
| Medium | 10:39 | #ff4400 | - |
| Heavy | 40:74 | #ff1100 | Tag applied: Hot |
| Obese | 75: | #850200 | Tag applied: Hot |
The trending items view shows featured trending items after basic moderation. Trending items are first captured by regularly snapshotting all users in social-media who've referenced vulnerabilities or have had a trending item pass moderation previously. The moderation process is a simple review to help cut-down the general noise and helps build logic to incorporate into ML algorithms in the near future.
The following table shows the current list of categories for trending items:
| Category | Description |
|---|---|
| Technical | A write-up, paper, tool, blog-post, exploit, etc. that has some technical security relevance. |
| Interesting | A link or statement that is interesting from a security perspective. |
| News | A link to security-relevant news. |
| General | A link to something non-security related, for example general world or IT news. |
| Humour | A funny link or statement, usually from someone in itsec. |
| None | No category assigned yet. |
* The categories should be considered a quick & dirty classifier for items.
For enquiries, bugs, ideas, and all the rest, please contact talkback@volvent.org.
Talkback