We are all inundated with information. Even in a Google-powered world, where it’s relatively easy to find any type of content that we may seek, it can sometimes still take significant effort to pore through the search results and find the specific pieces we are interested in.
For instance, say you’re interested in wearable tech. More specifically, you need to keep track of how the market is doing — but you’re not that interested in in-depth articles that explore the tech behind the devices. You want to filter out all that “unnecessary” content, and just read a certain type of article that covers the topic in a way that matters to you.
The Task + Data + Approach
We’re going to create a personalized content filter that only shows you articles that you’ve deemed are interesting to you. Let’s use the example we discussed above, and tune our filter to pick out articles that talk about the wearable device market specifically, while leaving out other content related to the topic. This is a simple classification problem, and in this case, there are two categories: Interesting and Not Interesting.
Now, we need a dataset. This is a CSV file of 30 articles discussing wearable devices, which I gathered from Wired, Financial Times, and other content providers after running a basic Google search on “wearable tech”. Next, we need to train the model with this unlabeled data to teach it what we care about, which we can quickly do using indico’s CrowdLabel. This tool enables you to quickly upload, label, and seamlessly train a model on your own data when used in conjunction with Custom Collections — all without writing any code.
If you’re thinking that dataset of 30 articles sounds too small to train a robust deep learning model, don’t worry. We can use such a small dataset because Custom Collections allows you to make use of Indico’s hundreds of thousands of rich feature embeddings through transfer learning.
Note: We’ll only be building the backend function (as in, the machine learning model) in this tutorial. This app would likely require some sort of frontend, but is not within the scope of this tutorial.
Using CrowdLabel
Let’s get started! First things first — go to CrowdLabel and sign into your indico account (or sign up if you don’t have one yet). If you already know how to use CrowdLabel, feel free to skip to the section that examines how well our filter performs.
Before uploading your dataset, make sure all your examples are placed in a single column, and each example should live in a separate cell. You should also give the column a heading, as the first example in your CSV file will automatically be interpreted as a column title.
- Specifying the number of labelers per task will set the minimum number of times an example must be labeled before it’s accepted as complete.
- Be sure to leave instructions for your labeling party under “Question” so they complete the task the way you want it. Make sure you are clear about the task. Here we specify, “Does the article talk about overall wearable tech market trends?” This way the labeler knows our criteria for what is interesting.
Now, let’s consider how the end product should work. The question we’re trying to answer as we read each article is “Is this article interesting or not?” We’ll allow labelers to select only one option, since the answers are mutually exclusive, by using radio buttons. Go ahead and fill out the different label options, and click “Create Dataset”.
Your Personalized Content Filter
Note that at the moment, you will need to run code to use your Custom Collection model, but you can do it in just a few lines of code. If you want to try running it, use this script and test dataset (these articles are different from the ones we used for training. You could also gather your own for testing if you prefer).
Otherwise, here are some accuracy metrics produced by the model:
{u'metrics': {u'recall': {u'no': 1.0, u'yes': 0.975}, u'class_accuracy': 0.9888888889, u'precision': {u'no': 0.9833333333000001, u'yes': 1.0}}}
metrics are typically used to measure the relevance of results for classification problems, such as this one. Basically, high precision indicates that the model found significantly more relevant results than irrelevant ones, and high recall tells us that the model found most of the possible relevant results. Looks like our model is really good at recognizing the sort of content we want to read (articles about the wearable tech market)! Now, to set the model to only return the “yes” (interesting) articles, we would just set a threshold probability in our code that tells the model to only show us articles that it’s confident we’ll like. If you want to be extremely strict, you could set it to yes = 0.9, but somewhere around 0.6 or 0.7 is probably good enough.
To give you a more tangible sense of how the model is working, here are some snippets of the articles the model deemed interesting vs. not interesting:
| Interesting | Not Interesting |
|---|---|
| Wristwear Dominates the Wearables Market While Clothing and Earwear Have Market-Beating Growth by 2021 The overall wearables market is expected to return to strong growth after a brief slowdown in 2016 that resulted from delayed launches of major platforms and notable vendors struggling to maintain pace. Looking ahead, new vendors, emerging form factors, and an expanded number of retail outlets will drive worldwide wearable device shipments from 102.4 million in 2016 to 237.5 million in 2021, a five-year compound annual growth rate (CAGR) of 18.3%, according to new data from the International Data Corporation (IDC) Worldwide Quarterly Wearable Device Tracker… |
Em-Sense Enabled Smartwatch Can Detect When You Touch a Doorknob The great promise of the smartwatch and its intelligent brethren is that they will unite everyday objects in a seamless web of connectivity. But there is a glaring hole in this plan: Our lives remain filled with countless dumb objects—door handles, lamps, pots and pans—that have no way of communicating with our smart objects. One solution is to make everything intelligent. A more discerning approach is to design objects that are clever enough to glean information from the dumb objects around them. This is the idea behind Em-Sense, a technology Carnegie Mellon’s future interfaces group developed with Disney Research… |
| Fitbit’s Dominance Diminishes But Wearable Tech Market Bigger Than Ever Fitbit has taken a hit with year-on-year Q4 sales but the wearable tech market is bigger than ever. They are the two main takeways from the latest IDC Worldwide Quarterly Wearable Device Tracker, which states that Fitbit shipments were down 22.7 percent in the fourth quarter of 2016: 6.5 million units compared to 8.4 million units in the same period in 2015… |
Athletes Turn to Implants to Take Training Into the Future With the rollout this season of RFID (radio-frequency identification) player-tracking technology in every NFL stadium—bottle-cap-sized chips are embedded in every set of shoulder pads—the league can keep tabs on all of its players the same way that businesses do their inventories. The system, provided by Zebra Technologies and still largely kept under wraps by the NFL, is geared partly toward improving the TV experience, but when that data is opened up to fans and teams, things could get really interesting… |
| Wearable Market in India Saw Shipment of 2.5 Million Units in 2016 Wearable market in India observed a shipment of 2.5 million units in 2016 with Goqii and Xiaomi dominating the entry-level segment, International Data Corporation’s (IDC) Worldwide Quarterly Wearable Device Tracker has said. Total wearable shipments in the fourth quarter of 2016 were 675,000 units, which included both smart wearables that can run third party apps and basic wearables, which cannot. Wearables shipments declined 19.6 per cent sequentially as basic wearables — that account to 86.4 per cent of total wearable — declined 23.9 per cent in the fourth quarter of 2016… |
Electronic Skin May Transform the Way We Interact With Tech It’s still just a proof-of-concept, but ideas for how to use skins like this one come pretty easily—think the Bionic Man, or Luke Skywalker’s prosthetic hand. But only recently has microfabrication tech caught up to humanity’s cyborg dreams. Now, every few months or so, a lab comes out with a new, different design: A bunch of sensors stuck to a glove! Circuitry printed on weird sticky paper! “We’re at a very nascent stage,” says Benjamin Tee, an e-skin researcher at the Institute of Materials Research and Engineering in Singapore… |
Next steps? Build a frontend for your app, and also hook up Text Tags to create an initial broad filter, before implementing this finer one. Questions? Reach out to us through that little chat bubble on the bottom righthand corner of your screen!
