I turned Panasonic’s built-in video intercom in my home with no IoT capabilities into one that connects to the AWS cloud, notifies me via Slack when guests come, plays a pre-recorded message, and unlocks the shared entrance door via Alexa and Amplify web app without changing the way it originally looked.

 

Story

In Japan, this kind of video intercom is very common in typical apartments with a single shared entrance. 

Here is how it works. A guest at the entrance presses my room number followed by the call button, which starts a video call. In my room, I’m notified by sound, check who is at the entrance by looking at the LCD screen, and start a conversation by pressing the talk button. If the guest is someone I expected, I unlock the entrance door by pressing the unlock button. The call will automatically end within several seconds after the door is unlocked, or I can manually end it by pressing the talk button again.

Video intercoms installed in Japanese apartments are very high-tech with numerous features, but sadly most of them don’t have any IoT capabilities, and since they’re integrated into the building security system,  it’s virtually impossible to replace them with smart doorbells that have been available recently. 

Some people have hacked intercoms in their apartments using ADCs, light sensors, or audio sensors to detect calls and using servo motors to perform unlock button presses. However, since I love hacks that people cannot tell IoT capabilities have been added just by looking from outside, this time I hacked the video intercom in my apartment (Panasonic’s SHVT18612WK), in a way that

  • Its original functionalities are intact and the hack is not visible from the outside.
  • Call detections and unlock operations are performed 100% electronically.
  • It is securely connected to the AWS cloud.
  • It plays pre-recorded messages.

 

The overall architecture of the hack and how it works

I used an ESP32 microcontroller with Mongoose OS installed and optocouplers to electronically detect calls and perform entrance door unlock operations, and a DFPlayerMini to play a pre-recorded message mp3 file generated by Amazon Polly.  

The overall architecture

 

Electronically detecting calls

When a video call is initiated from the entrance, the key-shaped green LED of the video intercom panel starts blinking. So, connect the primary side of an optocoupler in parallel to this LED and the secondary side to a GPIO and GND. By checking the state of the GPIO, you can detect calls. When a call is detected, ESP32 publishes a message to AWS IoT Core using the MQTT protocol and the topic, intercom/detect, and AWS Lambda is automatically executed to send you a notification via Slack.

Detecting calls and sending notifications

 

Electronically performing unlock button presses

Connect the primary side of an optocoupler to a GPIO and GND, and the secondary side in parallel to the unlock push button on the intercom circuit board. By setting the GPIO from L to H, then to L with slight delays in between, you can perform unlock operations without actually pressing the button. The same is true for the talk button. An AWS Amplify web app and an Alexa skill publish a message to AWS IoT Core using the topic, intercom/unlock, and ESP32 subscribed to the topic receives the message and does the job.

Unlocking the entrance door

 

AWS services used

AWS IoT Core

Used as an MQTT broker. You can set up Rules to automatically trigger subsequent AWS services. For example, a Lambda function can be configured so that it’s automatically executed when a message is published to a topic.  In this hack, if I need to automatically unlock the entrance door when ESP32 publishes a message to the topic, intercom/detect, I can set up a Rule that republishes a message to the other topic, intercom/unlock. Rules can be enabled and disabled from AWS CLI and Python scripts.

Copy to Clipboard

 

Copy to Clipboard

 

AWS Lambda

Used to send Slack messages when calls are initiated and to host a door unlocking Alexa skill.

AWS Amplify

Used to create a door unlocking web app.

Amazon S3

Used to host the Amplify web app.

Amazon CloudFront

Used as a CDN for the Amplify web app.

Amazon Cognito

Used to make the Amplify web app user authentication and user management secure and easy.

Alexa Skills Kit

Used to create the Alexa skill.

Amazon Polly

Used to create a welcome voice message, “Please come in”, which is played just before the entrance door is unlocked.

 

Real hack steps

Make sure to check any necessary laws and regulations in your country before attempting any of the steps below, and please do it at your own risk. 

 

Step 1. Expose the main PCB

First, remove the side covers and loosen screws. Make sure to turn the power switch off before removing the bundle of wires. This is because this kind of video intercoms comes with a built-in fire alarm and it goes off when those wires are removed, cut, and/or burnt with the power switch on.

 

 

 

 

Step 2. Look for hack spots on the PCB

For this hack, signal analysis is not necessary. Look for spots that can be easily hacked such as buttons, LEDs, and relays, etc.

Solder a ribbon cable to those spots. (cable color)

  1. Unlock standby LED + (Orange)
  2. Unlock standby LED – (Red)
  3. Unlock button (Blue)
  4. Talk button (Green)
  5. Unlock/Talk button shared GND (Yellow)

 

Step 3. Build an electrical circuit

On a mini breadboard, place an ESP32-DevKitC, optocouplers, and a DFPayerMini. To avoid stability issues as much as possible, use solid jumper wires instead of flexible ones. I found the VF of the key-shaped LED on the video intercom panel to be 3.0V, so I used IF=7.5mA (and R=250Ω) for the optocoupler for call detections, which is the optimal value for the operation under VF=1.15V according to its data sheet. If this IF is too big, the key-shaped LED won’t blink, and if it’s too small, the optocoupler won’t work. I used IF=15mA (and R=120Ω) for the optocouplers for talk and unlock button presses. For DFPlayerMini, connect Rx only because for this hack ESP32 doesn’t need to know when it has finished playing audio files, so Tx is not necessary.

 

The actual electric circuit looks the following. You could design a PCB for this.

 

Step 4. Write a code for ESP32

For this project, I used Mongoose OS, a very powerful IoT firmware development framework, so the application code (init.js) can be written in JavaScript. The full code is found in GitHub. First, install Mongoose OS to ESP32 and connect it to Wi-Fi using the following commands.

Copy to Clipboard

 

The following code block detects calls and publishes a message to topic1. Mongoose OS comes with a button handler to easily detect button presses. The ledPin is internally pulled-up, so it’s normally H and becomes L when a call is initiated. Timer.set() makes callState back to normal after 15 sec.

Copy to Clipboard

 

The following code block subscribes to topic2 and sequentially performs a talk button press, audio playback, an unlock button press, and a talk button press. What talk() and unlock() do is to set GPIOs from L to H, and then back to L with Sys.usleep() in between. I used 3 timers here because in Mongoose OS for this kind of use case, it is recommended to use Timer.set() instead of Sys.usleep() for stable operations.

Copy to Clipboard

Mongoose OS API reference is here.

 

Step 5. Provision ESP32 to AWS IoT Core

Since Mongoose OS contains the AWS IoT Device SDK for Embedded C and does everything necessary to connect your ESP32 to AWS IoT Core, what you need to do is to use IAM to get your Access Key ID and Secret Access Key pair ready and set up in your computer, and execute the following command.

Copy to Clipboard

 

Step 6. Do electrical work

BEFORE ANY ELECTRICAL WORK, TURN OFF THE AC POWER TO AVOID ELECTRICAL SHOCK.

Remove the base unit of the video intercom from the wall, remove the AC power cords, make a branch in the AC power line, attach a wall outlet with USB charging ports to the branch, and attach the AC power cords back to the base unit.

 

 

 

 

 

 

Step 7. Hide everything in the walls

Use a microUSB cable to power ESP32 and hide everything in the walls. Make sure to place the speaker close to the microphone, otherwise visitors cannot hear the audio message.

 

 

 

 

 

 

Future Directions

I want to make a PCB for this project. I also want to analyze the video signal, take it out from the board, and send it to the AWS Cloud to do fun stuff😁