“How is the weather (at home)? (বাসায় আবহাওয়া কেমন?)”,

My brother Quazi Obaida used to ask me this question whenever he was late to return home from outside, usually the soccer or cricket field. Like any other primary (and higher) school kids I guess we used to push our parents to check the boundaries, you know what I mean right? Now being a dad of two cheeky little ones and getting a taste of my own medicine, I must say our parents were extremely patient. 

And “weather” being the mood of our parents, আম্মা (mom) in particular, if s/he is angry about anything. Sunny and beautiful weather basically meant “no worries”, no scolding, no slap on the bottom even if that was actually due. Bad weather on the other hand meant a lot of things, and none of them you would look forward to. I mean who would want weather predictions the likes of “chance of rain“, “thunderstorm“, “severe thunderstorm with a chance of hailstorm” or even a freaking tornado. And you know how quickly these things escalate, right? Maybe if we stopped that cricket match at midday and came back home for lunch, we could’ve avoided the hailstorm and probably settled for the heavy rain.

But it wasn’t like I could call a bot remotely or query a web service and check the weather (mood) at home, right?

Neither could we predict if there will be a thunderstorm in advance. There was data (indicators) all around (how mum is answering my question in short answers, not smiling while talking to me, etc.) but we weren’t yet old enough to process that, let alone make predictions for the next two days.

But why I am writing about this? Well, an interesting thing happened a few months back, while driving home after picking my kid up from school, out of nowhere I asked him

“baba, how is the weather at home today?”

He seemed as confused as my voice sounded doubtful, thinking, what the heck did I just ask? It felt like deja-vu.

Now this is the bit you might find interesting. Can you predict how someone is feeling based on his/her behavior? I am sure you can, up to a point. If you actively look for signs you can probably draw conclusions.

Now back to the behavior bit, as we are all somehow connected to some sort of digital lifestyle, our daily behavior probably affects that digital footprint. You know, cat videos, YouTube clips, browsing through photo albums, listening to sad songs on loop, browsing profiles, etc., the usual ones. Even though there are no accurate one-to-one mappings between your mood and behavior (rather digital behavior), if you have enough data to map actual events, and know someone close enough to validate that information, you will start seeing patterns in that digital data.

You probably should have already guessed, the easiest way to get data about someone is to go through their log/ browsing history. Like any other mediocre Dev I searched stack-overflow and GitHub gist instead of writing something from scratch. At that point all I wanted to do was create a very simple PoC.

This is what I wanted to do in the first iteration. As a daily basis:

• Read my log every day and dump that into a csv with date and url;
• Analyze the log on 3 data points only (Music, Video, News/Article/Site, i.e. links other than music and video);
• Create a predefined taxonomy/vocab of weighted tags (this is completely biased with no automatic learning involved);
• Add the weighted value for the analysed log;
• Identify/create a relationship between total weight and the current mood, either happy or not happy (sad/annoyed/and what not, remember the Silicon Valley not hotdog app); and

• Validate, go back and fix the weighted taxonomy lookup table.

The above was my expectation, what happened was quite the opposite.

The first day (in reality 2nd day) there was not much data (forgot to not use incognito mode), and after 3/4 days I found that I’m very conscious about what I’m doing, so clearly it wasn’t the actual reflection at all.

I stopped my PoC and due to workload eventually forgot to check further.

After 2/3 weeks, out of curiosity I checked the log and noticed loads of interesting data. It turns out my better half had been using it instead of her laptop.

Now this became a moral dilemma. I definitely don’t want to go through what my better half was browsing, but at the same time I was curious whether my simple algorithm could predict how her day had been.

So I told her how anyone with access to the computer can see the browsing history and also showed her the incognito feature and how it doesn’t get logged (now here is a piece of advice, please don’t try to educate your better half and especially about the incognito mode, as you will have to answer more that you want to).

Now that it was off my chest (well sort of), I started validating the data and had to tweak the weight in the lookup table to adjust the actual result. And in some cases I had to revisit and change the value because actual results on certain days felt incorrect, after few days when the actual symptom was revealed. I wanted all this to be automated but it felt like I’m almost tweaking it every day and the patterns aren’t persistent. I had to rethink the approach and reevaluate my taxonomy table. I added another column relating to likely, highly likely and unsure (only used assertive path, didn’t even bother to go towards the non-assertive path). Now that seemed to simplify the manual involvement quite a bit.

After 3 to 3.5 months of data the prediction seems to be working most of the time (again, sort-of). The next step was to create a web service to query the data and I should have used Laravel but was being lazy and used one of local Drupal dev instance instead. I used feeds to periodically import the data and expose it using viewsjson. This was a very funny but pleasing feeling. From my driveway I can call the webservice running on my Dev environment and see the weighted score of happiness (or unhappiness or absence of data).

Now if I really want I can use any NLP provider like DialogFlow or Lex and literally ask “how is the weather at home today?“.

Or something like below:

Ok, I was joking… I didn’t end up doing that last bit. I imagine my smart phone assistant can remind me by saying, “hey dumbo, you better get some flowers for her today”. As ecstatic as it sounds from the PoC’s point of view, in reality it will sound very lame and pathetic.

It will be a sad day if I need to blindly rely on some lines of code rather than “looking at my better half” and sense if she’s not happy.

I don’t know how she does it, but I’m sure she doesn’t need to go through my logs to get it right. But I just wanted to see if my little setup will work, I was curious to see if it actually can flag whether I need to bring flowers home today.

So, how is the weather at your home today?

* First published on LinkedIn