Kino AI: Interactive Bot Streamer
An interactive TikTok Live robot that engages with the audience in real time, replying to comments using custom TTS voices and augmenting its responses with long term memory stored in a vector database powered by Zep, making interactions feel personal and human like.
Background Project|TikTok was blowing up with this trend called "checking spiritual guides" (khodam). People would drop their names in a host’s livestream, and the host acting like a shaman would “reveal” their spiritual guide just from the name. Honestly, I thought it was ridiculous… yet somehow the host kept getting loads of gifts. Social media never fails to surprise me. So, out of curiosity, I made a bot that does the exact same thing just to see if it could grab viewers’ attention too.
Kino AI Version 1|This is the exploration version, where I was simply figuring out what’s actually needed to build a livestream bot from scratch.
- Nextjs
Tech Stack V1 – Next.js detail|Build the bot public facing page.
- Tikfinity
Tech Stack V1 – Tikfinity detail|WebSocket tool for capturing TikTok livestream comments in real time.
- OpenAI
Tech Stack V1 – OpenAI detail|Handles the AI logic to answer questions or generate responses.
- Flow V1 – Step 1|Viewer comments on the TikTok livestream.
- Flow V1 – Step 2|Comment is captured by Tikfinity via WebSocket in real time.
- Flow V1 – Step 3|Next.js receives the event from Tikfinity and forwrads the viewer’s question which is their name.
- Flow V1 – Step 4|If there’s no ongoing process OpenAI generates response based on the question.
- Flow V1 – Step 5|Next.js updates the livestream with the generated response so other viewers can see it instantly.
Result V1|Boooooring, bounce rate is high because the bot wasn’t giving anything exciting. Without interesting or surprising responses, viewers quickly lost interest and scrolled away. On top of that, the async process could only handle one question at a time, and the OpenAI request often took 1–5 seconds, forcing viewers to stare at a loading screen, which only made them more impatient and complain like “why my comment isn’t being read!. I got only 1–5 viewers lol.
Kino AI Version 2|Learned from V1, I added queue system so the bot can process multiple comments in parallel no more blank screens. I also remove the “name only” rule so viewers can ask anything, introduced an anime style character for personality, and added TTS to make the stream feel alive.
- Nextjs
Tech Stack V2 – Next.js detail|Build the bot dashboard and public facing page.
- Tikfinity
Tech Stack V2 – Tikfinity detail|WebSocket tool for capturing TikTok livestream comments in real time.
- OpenAI
Tech Stack V2 – OpenAI detail|Answering question
- Applio
Tech Stack V2 – Applio detail|A simple, high quality voice conversion tool focused on ease of use and performance.
- Flow V2 – Step 1|Viewer comments on the TikTok livestream.
- Flow V2 – Step 2|Comment is captured by Tikfinity via WebSocket in real time.
- Flow V2 – Step 3|Next.js receives the event from Tikfinity and forwrads the viewer’s question.
- Flow V2 – Step 4|If (queue result + queue chat) is less than three and there’s no comment from the same viewer currently being processed, the chat is added to the queue, processed by OpenAI, and placed into the queue result.
- Flow V2 – Step 5|If there’s no ongoing TTS process (my GPU can’t handle more than one at the same time), Applio generates TTS from the OpenAI result and updates the queue result.
- Flow V2 – Step 6|Next.js gradually updates the UI with the queue results so other viewers can see them instantly.
TTS Exploration|Applio needs a clean voice dataset to train its custom TTS, so I used TopMediai to generate some sample paragraphs. However, the result wasn’t what I wanted, the silence between commas and periods was too long, which made the speech sound unnatural. To fix this, I used FFmpeg to automatically trim out the excessive silence and produce smoother audio.
- Topmediai
- Applio
- Applio + ffmpeg
Result V2 – Pros text|It caught a lot of attention my live viewers jumped to 50–100 per stream. I barely received any gifts or diamonds, but going from just 1–5 viewers is still solid progress. I also often gained over 100 new followers by the end of each stream.
Result V2 – Cons text|I had no idea how unpredictable and sometimes toxic anonymous viewer questions could be. I initially thought the questions would stay lighthearted and fun, but quickly realized that with anonymity comes a lot of unexpected or inappropriate prompts. I tried blocking certain keywords and even banning some viewers, but it didn’t work effectively. Inappropriate and sexual questions still slipped through. It became clear that simple filtering wasn’t enough to maintain the positive vibe I wanted, so I decided to change the bot’s character from a human figure to a robot figure instead.
Kino AI Version 3 – Intro|In this version I focused to create features to gather some diamonds
- V3 Features – Time Limit|Time Limit: The livestream runs for 10 minutes by default. Viewers can extend the time by sending gifts.
- V3 Features – Character Polling|Character Polling: Every 10 minutes, a polling modal appears. Viewers can boost their chosen option by 100x through sending gifts. If they don’t want to wait, they can trigger the modal instantly by sending a gift.
- V3 Features – Singing Polling|Singing Polling: Every 5 minutes, a singing polling appears. Viewers can boost their chosen option by 100x through sending gifts. If they don’t want to wait, they can trigger the poll instantly by sending a gift.
- V3 Features – Priority Viewers|Priority Viewers: Viewers who send gifts will be get faster response bypass the limit rules.
- Nextjs
Tech Stack V3 – Next.js detail|Builds the bot dashboard and public facing page.
- Tikfinity
Tech Stack V3 – Tikfinity detail|WebSocket tool for capturing TikTok livestream comments in real time.
- Applio
Tech Stack V3 – Applio detail|A simple, high quality voice conversion tool focused on ease of use and performance.
- Flowise
Tech Stack V3 – Flowise detail|A low-code visual tool for building and managing AI workflows, making it easier to connect APIs, models, and processing steps without complex coding.
- Zep Memory
Tech Stack V3 – Zep Memory detail|A long-term conversational memory service that stores and retrieves interactions, enabling the bot to remember context across sessions.
- OpenAI
Tech Stack V3 – OpenAI detail|Provides embeddings for memory storage and retrieval (via Zep) to generate responses to user.
- Three.js
Tech Stack V3 – Three.js detail|A JavaScript 3D library used to create and render interactive 3D visuals in the livestream overlay.
- Socket.IO
Tech Stack V3 – Socket.IO detail|Enables real time communication between my PC and external device, allowing me to manage and control the livestream remotely.
- Flow V3 – Step 1|Viewer posts a comment on the TikTok livestream.
- Flow V3 – Step 2|The comment is captured by Tikfinity via WebSocket in real time.
- Flow V3 – Step 3|Next.js receives the event from Tikfinity and forwards the viewer’s question.
- Flow V3 – Step 4 (blocked)|If the viewer is blocked, the chat will not be processed.
- Flow V3 – Step 5 (gifted viewer)|If the viewer sends a gift, the diamonds are counted and stored. When the viewer asks a question, they are treated as a priority viewer.
- Flow V3 – Step 6 (gifted chat)|If a gifting viewer asks a question, the chat is added to the queue, processed by Flowise, and placed into the queue result. One diamond is deducted from storage.
- Flow V3 – Step 7 (non-gifted queue)|If (queue result + queue chat) is less than three and no comment from the same viewer is currently being processed, the chat is added to the queue, processed by OpenAI, and placed into the queue result.
- Flow V3 – Step 8 (TTS)|If there’s no ongoing TTS process (My GPU can only handle one at a time), Applio generates TTS from the OpenAI result and updates the queue result.
- Flow V3 – Step 9 (UI update)|Next.js gradually updates the UI with queue results so other viewers can see them instantly.
Result V3|Although the viewer count doesn’t change much (around 50–100 viewers), I received a good number of gifts, most of them coming from the polling feature. On average, I earned between 50–900 diamonds per livestream, with each session lasting at least 3 hours. I still can’t compete with the “Shaman” I mentioned earlier, but I think this is good progress.
Blocker paragraph|TikTok has been stricter lately. I used to leave Kino AI running while I did other things even while sleeping but now they have a captcha challenge that appears at random times, which I can’t always complete. This first led to a 1 day ban, then a 1 week ban, and finally a 1 month ban. I think because of these frequent bans, my viewer count has dropped.