I turned Panasonic’s built-in video intercom in my home with no IoT capabilities into one that connects to the AWS cloud, notifies me via Slack when guests come, plays a pre-recorded message, and unlocks the shared entrance door via Alexa and Amplify web app without changing the way it originally looked.
In Japan, this kind of video intercom is very common in typical apartments with a single shared entrance.
Here is how it works. A guest at the entrance presses my room number followed by the call button, which starts a video call. In my room, I’m notified by sound, check who is at the entrance by looking at the LCD screen, and start a conversation by pressing the talk button. If the guest is someone I expected, I unlock the entrance door by pressing the unlock button. The call will automatically end within several seconds after the door is unlocked, or I can manually end it by pressing the talk button again.
Video intercoms installed in Japanese apartments are very high-tech with numerous features, but sadly most of them don’t have any IoT capabilities, and since they’re integrated into the building security system, it’s virtually impossible to replace them with smart doorbells that have been available recently.
Some people have hacked intercoms in their apartments using ADCs, light sensors, or audio sensors to detect calls and using servo motors to perform unlock button presses. However, since I love hacks that people cannot tell IoT capabilities have been added just by looking from outside, this time I hacked the video intercom in my apartment (Panasonic’s SHVT18612WK), in a way that
- Its original functionalities are intact and the hack is not visible from the outside.
- Call detections and unlock operations are performed 100% electronically.
- It is securely connected to the AWS cloud.
- It plays pre-recorded messages.
The overall architecture of the hack and how it works
I used an ESP32 microcontroller with Mongoose OS installed and optocouplers to electronically detect calls and perform entrance door unlock operations, and a DFPlayerMini to play a pre-recorded message mp3 file generated by Amazon Polly.
Electronically detecting calls
When a video call is initiated from the entrance, the key-shaped green LED of the video intercom panel starts blinking. So, connect the primary side of an optocoupler in parallel to this LED and the secondary side to a GPIO and GND. By checking the state of the GPIO, you can detect calls. When a call is detected, ESP32 publishes a message to AWS IoT Core using the MQTT protocol and the topic, intercom/detect, and AWS Lambda is automatically executed to send you a notification via Slack.
Electronically performing unlock button presses
Connect the primary side of an optocoupler to a GPIO and GND, and the secondary side in parallel to the unlock push button on the intercom circuit board. By setting the GPIO from L to H, then to L with slight delays in between, you can perform unlock operations without actually pressing the button. The same is true for the talk button. An AWS Amplify web app and an Alexa skill publish a message to AWS IoT Core using the topic, intercom/unlock, and ESP32 subscribed to the topic receives the message and does the job.
AWS services used
AWS IoT Core
Used as an MQTT broker. You can set up Rules to automatically trigger subsequent AWS services. For example, a Lambda function can be configured so that it’s automatically executed when a message is published to a topic. In this hack, if I need to automatically unlock the entrance door when ESP32 publishes a message to the topic, intercom/detect, I can set up a Rule that republishes a message to the other topic, intercom/unlock. Rules can be enabled and disabled from AWS CLI and Python scripts.
Used to send Slack messages when calls are initiated and to host a door unlocking Alexa skill.
Used to create a door unlocking web app.
Used to host the Amplify web app.
Used as a CDN for the Amplify web app.
Used to make the Amplify web app user authentication and user management secure and easy.
Alexa Skills Kit
Used to create the Alexa skill.
Used to create a welcome voice message, “Please come in”, which is played just before the entrance door is unlocked.
Real hack steps
Make sure to check any necessary laws and regulations in your country before attempting any of the steps below, and please do it at your own risk.
Step 1. Expose the main PCB
First, remove the side covers and loosen screws. Make sure to turn the power switch off before removing the bundle of wires. This is because this kind of video intercoms comes with a built-in fire alarm and it goes off when those wires are removed, cut, and/or burnt with the power switch on.
Step 2. Look for hack spots on the PCB
For this hack, signal analysis is not necessary. Look for spots that can be easily hacked such as buttons, LEDs, and relays, etc.
Solder a ribbon cable to those spots. (cable color)
- Unlock standby LED + (Orange)
- Unlock standby LED – (Red)
- Unlock button (Blue)
- Talk button (Green)
- Unlock/Talk button shared GND (Yellow)
Step 3. Build an electrical circuit
The actual electric circuit looks the following. You could design a PCB for this.
Step 4. Write a code for ESP32
The following code block detects calls and publishes a message to topic1. Mongoose OS comes with a button handler to easily detect button presses. The ledPin is internally pulled-up, so it’s normally H and becomes L when a call is initiated. Timer.set() makes callState back to normal after 15 sec.
The following code block subscribes to topic2 and sequentially performs a talk button press, audio playback, an unlock button press, and a talk button press. What talk() and unlock() do is to set GPIOs from L to H, and then back to L with Sys.usleep() in between. I used 3 timers here because in Mongoose OS for this kind of use case, it is recommended to use Timer.set() instead of Sys.usleep() for stable operations.
Mongoose OS API reference is here.
Step 5. Provision ESP32 to AWS IoT Core
Since Mongoose OS contains the AWS IoT Device SDK for Embedded C and does everything necessary to connect your ESP32 to AWS IoT Core, what you need to do is to use IAM to get your Access Key ID and Secret Access Key pair ready and set up in your computer, and execute the following command.
Step 6. Do electrical work
BEFORE ANY ELECTRICAL WORK, TURN OFF THE AC POWER TO AVOID ELECTRICAL SHOCK.
Remove the base unit of the video intercom from the wall, remove the AC power cords, make a branch in the AC power line, attach a wall outlet with USB charging ports to the branch, and attach the AC power cords back to the base unit.
Step 7. Hide everything in the walls
Use a microUSB cable to power ESP32 and hide everything in the walls. Make sure to place the speaker close to the microphone, otherwise visitors cannot hear the audio message.
I want to make a PCB for this project. I also want to analyze the video signal, take it out from the board, and send it to the AWS Cloud to do fun stuff😁