Beating the Turing test

Two and a half years ago I had a shower-thought: "Can I build a space online that feels lived-in?"

At that time I was living in the historic center of Florence, Italy and was fascinated by the incredible door-knockers you can find on almost every door. I had also recently rebooted my website and wasn't very happy with how it turned out.

So with that on my mind, the thought soon crystallized into a virtual door that I "lived" behind, connected directly to my phone. I spent a week building it and replaced my website in April of '17.

Since then 15,238 people have knocked at my door and I've chatted with 4,249 of them.

This is how it worked: When someone visited my website they were presented with a 3D door knocker that when hit would send a notification to me. If I responded the door would open and reveal a chat interface and we'd have a conversation.

Are you real? I don't believe you, prove it!

That's by far the most common reaction and my response is usually along the lines of "Sure, but how?". It's been very interesting to see how different people approach the problem of asserting that someone is human over a text chat. This is what the Turing test is all about, named after and developed by Alan Turing in 1956 in his quest to understand intelligence.

A popular strategy is asking about current news, usually, something involving US politics. Another one is to "say something only a human would", which is a really fun challenge to try to come up with something creative and clever enough to satisfy.

People familiar with machine learning are the easiest to convince because they know the current limitations of A.I. and what weaknesses to test. One of the best test questions I got was: What's your most recent memmory involving pine apple?

Notice the deliberate grammar and spelling mistakes, not only would a model have to correct the spelling mistake it would also need to understand that the object is a pineapple, not pine and apple. Not to mention the level of intelligence needed to understand and respond to the question once understood correctly.

The current Loebner Prize winner Mitsuku fails pretty bad when given that question:

me: Hi
mitsuku: Hi there.
me: What's your most recent memmory involving pine apple?
mitsuku: The one that is most recent would be the latest.
me: Tell me about it
mitsuku: latest sounds like a type of perfume to me

You could argue that a question like this goes against the spirit of a traditional Turing test and that it gives away a Turing test is being administered. But there is no arguing its effectiveness.

You've made my day!

For the most part, I pass the test and when I do the response has been overwhelmingly positive. I've chatted with so many interesting people who stumble across my site and It's been very fulfilling to share in the delight my visitors get from the experience. Initially, I planned to only run the experiment for a couple of months but ended up running it for over two years.

I also think I captured my original intent of "creating a space that felt lived in", many people feel like they are intruding and apologize for wasting my time and needlessly knocking on my door (to which my response is always that they are not intruding and are welcome guests).

Be nice to robots

Some guests were not that gracious though. When visitors were rude even after accepting that they were talking to a human or they were just exceedingly stubborn I gave myself license to have a bit of fun at their expense. In addition to being a chat interface from my end, I could run commands, slam the door, play sounds and music and even run arbitrary JavaScript in their browser.

To unbelievers I sometimes role-played an evil AI trying to break out of its prison, messing with them by running JavaScript commands, showing dialogs like Allow root access? or asking for WebCam permissions. I think I managed to truly freak some people out... if any of you happen to read this I apologize, but you can't deny it was a unique experience ;)

Stats and technical details for nerds

Since the start I've had 22,605 visitors, 15,238 of them knocked, 4,249 got a response and I've had 1,882 meaningful conversations where both parties sent 5 or more messages to each other.

That works out to me opening the door 5 times and having 2 conversations per day, for the last two and a half years.

I managed to open the door for 28% of the time which is more than I expected but I can't help but feeling some FOMO for the 72% I missed considering how many opportunities and awesome people that door opened up to.

Visitors are counted based on when someone loaded in the WebGL scene and connected to my chat service, total unique visitors as measured by analytics is over to 200,000 for the period. Which I think says something about how many bots are online and how much salt you should put on web analytics numbers.

The system I set up to make this happen is a backend connected to a Slack workspace speaking with clients over WebSockets using my own wsrpc library. Each visitor gets their own channel that is automatically archived when they leave the page.

Using Slack was a good choice, it allows you to set hours for when notifications should be allowed to come through and have a "Snooze notifications for X hours" feature, both of which were invaluable for this particular application.

The frontend is built with wintersmith, it uses three.js to draw the knocker and cannon.js for the knocking physics simulation, it even measures the impact velocity so I can see how hard visitors are knocking, because why not.

Initially, I tried to recreate the knocker faithfully with high-quality textures and model but I wasn't able to get good load times frame rates on mobile so I created the low-poly look and ended up liking it better so went with that across the board.

Final thoughts

This website has been my companion for the last two and a half years and I'm both sad and relieved to see it go. I've had so many great conversations with awesome people from all over the world that I never would have spoken with otherwise.

I've also been surprised at how little abuse I got exposing myself to anonymous strangers on the internet. In some ways, it has cemented a belief in me that all people are fundamentally good even though there likely is some selection bias at work in the type of person that finds their way to my website.

On the other hand I'm glad to get rid of this ultimate procrastination machine, at of the time of writing the knocker is still up on johan-nordberg.com but I'm planning to replace it soon – hopefully with something less time-consuming.