The brain PC · Chapter 28 · Time: 3 hours · Level: Intermediate · Status: Partly test-built
Optional. Buddy's animated 3D face on bigbuddy's touchscreen in a Chromium kiosk that the voice loop drives over the DevTools protocol, a hotkey and touch controls, music and movies from Plex in two podman containers, and voice control of the lights, the TV and the Evo 150 amplifier. The robot works without any of it.
This chapter is optional. It gives Buddy a face on bigbuddy's touchscreen and lets him play music, play movies and
switch the room's lights, TV and amplifier. Nothing the robot needs depends on it: the robot drives, hears and
speaks without a screen, and since 2026-10-07 it runs headless by the owner's decision. Build it if you want the
face and the media; skip to chapter 29 if not.
All of it runs on bigbuddy, inside or beside Buddy's voice loop from chapter 26. It was built between 2026-09-28
and 2026-10-01 and read back on 2026-10-07. A few steps were done by the owner and not recorded command by command
(Chromium install, autologin, the Plex and Home Assistant tokens); those are marked. One known bug, the full-screen
restore after widget mode, was fixed by hand on 2026-10-07 and is not in the code.
| Port | Bound to (live 2026-10-07) | What |
|---|---|---|
| 8290 | 0.0.0.0 |
the face server, inside buddy-voice (the repo binds 127.0.0.1; see below) |
| 9222 | 127.0.0.1 |
Chromium's DevTools port, used by media_screen.py |
| 8281 | 127.0.0.1 |
the jukebox container (Plex music) |
| 8282 | 127.0.0.1 |
the video container (Plex movies) |
bigbuddy has two displays. DP-1 is the WingCool touchscreen, the primary display, where the face lives. HDMI-A-1
is the Samsung TV in the den. The touchscreen ran at 1280x720 on 2026-10-07; voice/README.md still says
1920x1080 at scale 1.35, from 2026-10-01.
You need chapter 26 working: buddy-voice.service running, the voice venv in ~/voice/venv (it has requests and
websockets, which media_screen.py uses), and the audio set-up of chapter 27, because the kiosk and the music
play into buddy_aec_sink.
Packages. Chromium comes from Fedora (dnf transaction 43, 2026-09-30, dnf install -y chromium). mpv was already
installed by the owner (2026-08-07), and podman is part of Fedora's base image. bigbuddy's sudo asks for a
password, so the owner runs dnf himself.
On bigbuddy:
sudo dnf install -y chromium
rpm -q chromium podman
Check
Versions on 2026-10-07:chromium-154.0.8037.57-1.fc44.x86_64 podman-5.8.7-1.fc44.x86_64
The face needs a desktop session, so bigbuddy logs in by itself. The live /etc/plasmalogin.conf has:
On bigbuddy:
[Autologin]
Session=plasma.desktop
User=burgerbarn
The docs differ: voice/README.md ("Boot (2026-10-01)") says bigbuddy has no autologin. The live file says it
has, and the journal shows the autologin session opening at the 2026-10-04 boot.
Not recorded: how autologin was set
The file is dated 2026-07-31, before this project, andrpm -Vreports it modified. The command or settings
page the owner used is not in the record.
voice/face_server.py is a small web server built only from Python's standard library. It runs inside the voice
loop's process: buddy_voice.py calls face.start(); face.state('idle') once at start-up, then reports every
change (listening, thinking, working on a tool, speaking). The page voice/face/face3d.html draws the face with
three.js (vendored in voice/face/vendor/three.module.js, no internet needed) and listens for those changes.
New idea: Server-Sent Events
A web page normally asks the server for something and gets one answer. With Server-Sent Events (SSE) the
page opens one HTTP request that never ends, and the server writes a linedata: {...}into it whenever
something happens. It is one-way, server to page, and needs nothing but HTTP. The face page opens
/eventsand redraws on every message.
The file is 289 lines. Its parts:
| Part | What it does |
|---|---|
_publish(ev) |
puts one event into every connected page's queue |
state(name, text) |
sets Buddy's state; also tells the robot (/heard event state) so the robot's body language can follow |
level(x) |
the microphone level while listening (eyes, a small mouth ripple) |
speak(a, sr) |
turns Buddy's exact Kokoro samples into 20 frequency bands (90 to 7000 Hz), 30 times a second, timed to playback: the mouth |
music(on) |
music mode: the mouth becomes the live spectrum of whatever plays into buddy_aec_sink (a parec tap on its monitor) |
_H.do_GET |
/face (and /, /face3d), /face.js, /overlay.js, /vendor/three.module.js, /events, and /robot, /robot/status, /robot/frame.jpg, /robot/stream.mjpg relayed to the robot API with the token |
_H.do_POST |
/mode (screen modes), /listen (a tap on the face), /music (touch gestures), /robot/do (robot panel buttons: stop, come, home, look) |
start() |
starts the server in a background thread |
The heart of it is the event fan-out. Every page that opens /events gets its own queue; _publish writes to all
of them:
On bigbuddy:
def _publish(ev):
data = ('data: ' + json.dumps(ev) + '\n\n').encode()
with _lock:
for q in list(_clients):
try:
q.put_nowait(data)
except queue.Full:
pass
and the /events handler sends the current state first, then whatever arrives, with a comment line every 15 s
so the connection never looks dead:
On bigbuddy:
if p == '/events':
q = queue.Queue(maxsize=200)
with _lock:
_clients.append(q)
self.send_response(200)
self.send_header('Content-Type', 'text/event-stream'); self.send_header('Cache-Control', 'no-cache')
self.send_header('Access-Control-Allow-Origin', '*'); self.end_headers()
try:
for ev in (_last_state, _music):
self.wfile.write(('data: ' + json.dumps(ev) + '\n\n').encode())
self.wfile.flush()
while True:
try:
self.wfile.write(q.get(timeout=15))
except queue.Empty:
self.wfile.write(b': ping\n\n')
self.wfile.flush()
except (BrokenPipeError, ConnectionResetError, OSError):
pass
finally:
with _lock:
if q in _clients:
_clients.remove(q)
return
Copy the voice code and the face folder to bigbuddy. This is the copy command from the record, with the repo as the
source:
On your laptop:
R=~/CCode/rosorin-pro/voice; scp -q $R/face_server.py $R/buddy_voice.py $R/media_screen.py bigbuddy:voice/ && scp -q -r $R/face bigbuddy:voice/
ssh bigbuddy 'systemctl --user restart buddy-voice.service'
On bigbuddy:
curl -s -o /dev/null -w "face %{http_code}\n" 127.0.0.1:8290/face
timeout 3 curl -sN 127.0.0.1:8290/events | head -1
Check
Output from 2026-09-29:face 200 data: {"t": "state", "state": "idle", "text": ""}
The repo's start() listens on loopback only:
On bigbuddy:
def start():
srv = ThreadingHTTPServer(('127.0.0.1', PORT), _H)
srv.daemon_threads = True
threading.Thread(target=srv.serve_forever, daemon=True).start()
return srv
The live file on bigbuddy was edited by hand on 2026-10-06 (backup ~/voice/face_server.py.bak-20261006) so the
robot's touchscreen could read the face events over the LAN. Line 286, live:
On bigbuddy:
srv = ThreadingHTTPServer((os.environ.get('BUDDY_FACE_HOST', '0.0.0.0'), PORT), _H) # 2026-10-06: the robot's touchscreen shows the face over the LAN
The robot client is robot_face/face_fb.py, run by rosorin-face.service, which draws the face straight into the
robot's framebuffer from http://192.168.1.111:8290/events. Since 2026-10-07 the robot is headless and
rosorin-face is disabled (scripts/ops/robot_mode.sh light; robot_mode.sh face turns it back on).
Safety: the face server on the LAN can start the robot's wheels
POST /robot/dowith bodycomeorhomeis relayed to the robot API with Buddy's token and starts
rosorin-task@comeorrosorin-task@home, which drive the robot. The face server checks no token of its
own; its docstring says "Server is loopback-only". Live it listens on0.0.0.0, and bigbuddy's firewall zone
(FedoraWorkstation) opens TCP 1025-65535 to the LAN, so any device on the LAN can send that request. With the
robot headless, nothing needs the LAN binding. Keep the repo's127.0.0.1when you build, or set
BUDDY_FACE_HOST=127.0.0.1forbuddy-voice. This was not changed on the live machine.
The face, the movie player and (until 2026-10-01) the jukebox are web pages. They show in a Chromium window with
no browser controls (a kiosk), and Buddy's code drives that window from outside: opening tabs, navigating,
calling functions in the page. voice/media_screen.py does all of it.
New idea: the Chrome DevTools Protocol (CDP)
Started with--remote-debugging-port=9222, Chromium accepts commands on that port. Plain HTTP lists and
manages tabs:/json(all tabs),/json/new?<url>,/json/activate/<id>,/json/close/<id>,
/json/version. Each tab also has a websocket. Over it you send JSON commands such asPage.navigateor
Runtime.evaluate(run JavaScript in the page) and get JSON answers. This is how test tools drive browsers.
Here it lets the voice loop start music in a page, switch tabs or read the page size, without anyone
touching the screen.
Why Chromium and not Firefox
The first kiosk (2026-09-28) was Firefox. On the 4K TV it drew the 3D face at 3.3 frames per second (12 after
reducing the texture). The same page in Chromium ran at 127 fps; drawing the face itself takes 1.1 ms per
frame, so Firefox's WebGL compositing on this Wayland desktop was the bottleneck (voice/README.md
"3D face",docs/decisions.md2026-09-30). Firefox and its profile were removed.
ensure_browser() starts the kiosk if it is not already running. It uses systemd-run --user to start Chromium
as a transient user unit called buddy-screen: a unit that exists only while it runs, so systemd tracks and
stops it, and nothing is installed on disk:
On bigbuddy:
def ensure_browser():
"""Start the kiosk Chromium (own profile, transient user unit, window class buddy-screen) if not running."""
if not _running():
os.makedirs(PROFILE, exist_ok=True)
subprocess.run(['systemctl', '--user', 'reset-failed', f'{UNIT}.service'], stderr=subprocess.DEVNULL)
subprocess.run(['systemd-run', '--user', '-q', f'--unit={UNIT}', '--collect',
f'--setenv=PULSE_SINK={SPK}', # sound to HDMI, not the default (Komplete Audio 6)
'--setenv=WAYLAND_DISPLAY=wayland-0',
shutil.which('chromium-browser') or shutil.which('chromium'), '--kiosk',
'--ozone-platform=wayland', '--class=buddy-screen', # KWin finds it by this class (hotkey, modes)
f'--user-data-dir={PROFILE}', f'--remote-debugging-port={PORT}',
'--autoplay-policy=no-user-gesture-required', '--no-first-run', '--noerrdialogs',
'--disable-session-crashed-bubble', '--disable-features=Translate,MediaRouter',
# faceplay plays in a background tab behind the face: never throttle/suspend hidden tabs/windows
# (2026-10-01: the jukebox never requested the song while hidden)
'--disable-background-media-suspend', '--disable-renderer-backgrounding',
'--disable-background-timer-throttling', '--disable-backgrounding-occluded-windows',
'--password-store=basic', '--hide-crash-restore-bubble', 'about:blank'], check=True)
for _ in range(60):
try:
requests.get(f'http://127.0.0.1:{PORT}/json/version', timeout=1)
return
except Exception:
time.sleep(0.5)
raise RuntimeError('kiosk Chromium did not open its DevTools port')
What the flags are for:
| Flag | Why |
|---|---|
PULSE_SINK=buddy_aec_sink |
the kiosk's sound goes into Buddy's sink, to the Evo 150 (the code comment still says HDMI; the sink now forwards to the Evo) |
--kiosk, --noerrdialogs, --disable-session-crashed-bubble, --hide-crash-restore-bubble, --no-first-run |
full screen, no bars, no pop-ups after a crash or on first start |
--ozone-platform=wayland, --class=buddy-screen |
native Wayland window with a fixed class name, so KWin scripts and the hotkey can find it |
--user-data-dir=~/.buddy-screen-chromium |
its own profile, separate from any browser you use |
--remote-debugging-port=9222 |
the CDP port (Chromium listens on 127.0.0.1) |
--autoplay-policy=no-user-gesture-required |
sound may start without a click |
the four --disable-background... / --disable-renderer-backgrounding flags |
a tab hidden behind the face keeps playing (on 2026-10-01 a hidden jukebox tab never fetched its song) |
--disable-features=Translate,MediaRouter, --password-store=basic |
no translate bar, no cast menu, no keyring prompt |
Page commands go through _eval(), which runs JavaScript in the page with userGesture: True, so the page treats
it like a click (autoplay, Web Audio and full screen need one):
On bigbuddy:
async def _eval(c, body):
"""Run an async JS body in the page WITH a user gesture (autoplay / WebAudio / fullscreen); returns its value."""
r = await c.cmd('Runtime.evaluate', {'expression': f'(async () => {{ {body} }})()', 'awaitPromise': True,
'returnByValue': True, 'userGesture': True})
if 'exceptionDetails' in r:
raise RuntimeError(r['exceptionDetails'].get('text', 'page error'))
return r.get('result', {}).get('value')
The rest of the file (463 lines) builds on these: show_face() (the idle screen: face in front unless music or a
movie is up), play_movie(), control(), set_style(), set_fullscreen(), and set_mode() (next section).
Run it by hand with its own command line: ~/voice/venv/bin/python ~/voice/media_screen.py prints the usage.
Check the kiosk from bigbuddy: list the tab and read the page size against the screen size.
On bigbuddy:
cd ~/voice && venv/bin/python - <<'PY'
import media_screen as m
p = m._pages()[0]
print(p['url'], m._run(p, lambda c: m._eval(c, "return [innerWidth, innerHeight, outerWidth, outerHeight, screen.width, screen.height]")))
PY
Check
_pages()starts the kiosk if it is not running. Output on 2026-10-07 on the 1280x720 touchscreen:http://127.0.0.1:8290/face [1280, 720, 1280, 720, 1280, 720]The page fills the whole screen. If the first two numbers are smaller than the last two, the window has bars;
see "The full-screen restore problem" below.
If it fails
RuntimeError: kiosk Chromium did not open its DevTools port: Chromium did not start, usually because
there is no desktop session yet (no Wayland display). Check that you are logged in (autologin) and run
journalctl --user -u buddy-screen -n 20.- Moving the running kiosk from the HDR TV (scale 2) to the SDR touchscreen (scale 1.35) on 2026-10-01 drew
the page, then went black. Start it on the right display instead of moving it: the touchscreen is the
primary display, so the kiosk opens there. Window moving is off in the code (BUDDY_PLACEdefault 0).
buddy-voice starts at boot (linger) and calls show_face() once, but at that moment the desktop may not be up
yet. A second, one-shot user unit tries again when the Plasma session starts. Create
~/.config/systemd/user/buddy-face-screen.service (repo voice/buddy-face-screen.service, identical live):
On bigbuddy:
[Unit]
Description=Buddy's face on the Den TV once the desktop session is up (media_screen.show_face; no-op if running)
PartOf=graphical-session.target
After=graphical-session.target
[Service]
Type=oneshot
WorkingDirectory=%h/voice
ExecStartPre=/bin/sleep 5
ExecStart=%h/voice/venv/bin/python -c "import media_screen as ms; ms.show_face()"
[Install]
WantedBy=graphical-session.target
graphical-session.target is reached when the desktop session has started; WantedBy it means "start this then".
The 5-second sleep gives the compositor time to accept windows.
On your laptop:
scp -q ~/CCode/rosorin-pro/voice/buddy-face-screen.service bigbuddy:.config/systemd/user/ && ssh bigbuddy 'systemctl --user daemon-reload && systemctl --user enable buddy-face-screen.service'
Check
Created symlink '/home/burgerbarn/.config/systemd/user/graphical-session.target.wants/buddy-face-screen.service' → '/home/burgerbarn/.config/systemd/user/buddy-face-screen.service'.On 2026-10-07 the boot path was tested without a reboot: stop the kiosk, start this unit, read the page size.
On bigbuddy:
systemctl --user stop buddy-screen.service; sleep 2 systemctl --user start buddy-face-screen.service; echo "boot unit exit: $?"gave
boot unit exit: 0and thenhttp://127.0.0.1:8290/face [1280, 720, 1280, 720, 1280, 720]: full
screen, no bars.
The docs differ: docs/lessons.md (2026-10-01) says this unit was disabled at 15:04 that day. Live it is
enabled; its link is dated 15:13 the same day, and it ran at the 2026-10-04 boot.
The kiosk has four modes. All of them are KWin operations on the window with class buddy-screen.
| Mode | Effect | How to get there |
|---|---|---|
buddy |
kiosk restored and in front | voice "buddy mode", POST /mode buddy, Meta+Shift+B |
desktop |
kiosk minimized, Buddy keeps listening | voice "desktop mode", the "Desktop" button on the face, Meta+Shift+B |
widget |
out of full screen, a quarter-size window bottom right, kept above other windows | the "Widget" button on the face |
full |
back to full screen | the "Full" button on the widget, POST /mode full |
New idea: KWin scripts
KWin is KDE's window manager. It runs small JavaScript programs that can see and change every window:
workspace.windowList()lists them, and each window has properties such asminimized,fullScreen,
keepAboveandframeGeometry. A script can be installed permanently (with a keyboard shortcut) or loaded
once over D-Bus, run, and unloaded.
set_mode() writes a one-line script for the chosen mode, loads it into KWin over D-Bus with gdbus, runs it and
unloads it:
On bigbuddy:
_FIND = "const w = workspace.windowList().find(w => w.resourceClass === 'buddy-screen');\n"
_MODE_JS = {
'buddy': "if (w) { w.minimized = false; workspace.activeWindow = w; }",
'desktop': "if (w) w.minimized = true;",
# 25 % widget (owner 2026-09-30): out of full screen, bottom-right quarter of the work area, kept above other
# windows; resizable (KDE: Alt+right-drag, or the window edges)
'widget': ("if (w) { const a = workspace.clientArea(KWin.MaximizeArea, w); w.minimized = false; w.fullScreen = false;"
" w.keepAbove = true; w.frameGeometry = {x: a.x + a.width / 2, y: a.y + a.height / 2,"
" width: a.width / 2, height: a.height / 2}; workspace.activeWindow = w; }"),
'full': "if (w) { w.minimized = false; w.keepAbove = false; w.fullScreen = true; workspace.activeWindow = w; }",
}
On bigbuddy:
def set_mode(mode):
"""'buddy' = screen in front, 'desktop' = minimized, 'widget' = a 25 % window bottom-right (kept above),
'full' = back to full screen. One-shot KWin script over D-Bus; the hotkey (Meta+Shift+B) is the installed KWin
script kwin/buddytoggle."""
if mode not in _MODE_JS:
raise ValueError(mode)
if mode != 'desktop' and not _running():
show_face()
path = os.path.expanduser('~/.cache/buddy-mode.js')
with open(path, 'w') as f:
f.write(_FIND + _MODE_JS[mode])
call = ['gdbus', 'call', '--session', '--dest', 'org.kde.KWin']
out = subprocess.run(call + ['--object-path', '/Scripting', '--method', 'org.kde.kwin.Scripting.loadScript', path, 'buddymode'],
capture_output=True, text=True).stdout
sid = ''.join(c for c in out if c.isdigit())
if sid:
subprocess.run(call + ['--object-path', f'/Scripting/Script{sid}', '--method', 'org.kde.kwin.Script.run'], capture_output=True)
time.sleep(0.3)
subprocess.run(call + ['--object-path', '/Scripting', '--method', 'org.kde.kwin.Scripting.unloadScript', 'buddymode'], capture_output=True)
return {'buddy': 'Buddy mode.', 'desktop': 'Desktop mode.', 'widget': 'Widget mode.', 'full': 'Full screen.'}[mode]
The face page's buttons call it through the face server, and so can you:
On your laptop:
ssh bigbuddy 'curl -s -X POST --data full http://127.0.0.1:8290/mode'
The server answers 204 with no body and accepts only desktop, buddy, widget and full (anything else is
400).
widget takes the window out of full screen. full only sets KWin's fullScreen flag. On 2026-10-07, after the
owner used widget mode on the touchscreen, full left Chromium as a normal maximized window with its tab strip and
address bar showing. The page size read back:
http://127.0.0.1:8290/face [1280, 633, 1280, 720]
The window covered 1280x720, but the page got only 633 pixels of height. The fix was to tell Chromium itself to go
full screen through CDP: find its window for the face tab, then Browser.setWindowBounds with
windowState: fullscreen. This is the command that was run by hand:
On your laptop:
ssh bigbuddy 'bash -s' <<'EOF'
cd ~/voice && venv/bin/python - <<'PY'
import asyncio, json, requests, websockets
import media_screen as m
page = m._pages()[0]
bws = requests.get('http://127.0.0.1:9222/json/version', timeout=3).json()['webSocketDebuggerUrl']
async def go():
async with websockets.connect(bws, max_size=None, origin=None) as ws:
c = m.Cdp(ws)
win = await c.cmd('Browser.getWindowForTarget', {'targetId': page['id']})
print('before', win)
await c.cmd('Browser.setWindowBounds', {'windowId': win['windowId'], 'bounds': {'windowState': 'fullscreen'}})
await asyncio.sleep(1.0)
print('after', await c.cmd('Browser.getWindowBounds', {'windowId': win['windowId']}))
asyncio.run(go())
print(m._run(page, lambda c: m._eval(c, "return [innerWidth, innerHeight, screen.width, screen.height]")))
PY
EOF
Browser.* commands go to the browser-wide websocket (webSocketDebuggerUrl from /json/version), not a tab's.
Check
before {'windowId': 1001610574, 'bounds': {'left': 0, 'top': 0, 'width': 1280, 'height': 720, 'windowState': 'maximized'}} after {'bounds': {'left': 0, 'top': 0, 'width': 1280, 'height': 720, 'windowState': 'fullscreen'}} [1280, 720, 1280, 720]
Not in the code
media_screen.set_mode('full')still only sets KWin's flag, so going from widget back to full shows the bars
again until you run the command above, restart the kiosk, or reboot. The code change (the same CDP call inside
set_mode('full')) was offered on 2026-10-07; the owner declined it and asked only that the machine start up
full screen, which it does.voice/README.mdsays widget and full were verified both ways on the TV on
2026-09-30; on the touchscreen on 2026-10-07 the way back failed. What changed between the two is not known.
A permanent KWin script toggles between the face and the desktop. It is two files in the repo,
voice/kwin/buddytoggle/metadata.json:
On bigbuddy:
{
"KPlugin": {
"Id": "buddytoggle",
"Name": "Buddy screen toggle",
"Description": "Meta+Shift+B flips between Buddy's face (kiosk Firefox, app id buddy-screen) and the desktop",
"Version": "1.0",
"License": "MIT",
"EnabledByDefault": true
},
"X-Plasma-API": "javascript",
"KPackageStructure": "KWin/Script"
}
and voice/kwin/buddytoggle/contents/code/main.js:
On bigbuddy:
// Buddy mode <-> desktop mode: minimize / restore the kiosk window (Firefox launched with --name buddy-screen).
function kiosk() { return workspace.windowList().find(w => w.resourceClass === 'buddy-screen'); }
function toggle() {
const w = kiosk();
if (!w) return;
if (w.minimized || workspace.activeWindow !== w) { w.minimized = false; workspace.activeWindow = w; }
else w.minimized = true;
}
registerShortcut('BuddyScreenToggle', 'Buddy: toggle face / desktop', 'Meta+Shift+B', toggle);
The comments still say Firefox. The script finds the window by its class, buddy-screen, which the Chromium kiosk
sets with --class, so it works unchanged. Meta+B was already taken by the power profile.
Install it (commands from 2026-09-29): copy it into KWin's script folder, back up kwinrc, enable the plugin, tell
KWin to reload, and check.
On your laptop:
ssh bigbuddy 'mkdir -p ~/voice/kwin ~/.local/share/kwin/scripts'; scp -qr ~/CCode/rosorin-pro/voice/kwin/buddytoggle bigbuddy:voice/kwin/
On bigbuddy:
cp -r ~/voice/kwin/buddytoggle ~/.local/share/kwin/scripts/
cp ~/.config/kwinrc ~/.config/kwinrc.bak-buddytoggle
kwriteconfig6 --file kwinrc --group Plugins --key buddytoggleEnabled true
gdbus call --session --dest org.kde.KWin --object-path /KWin --method org.kde.KWin.reconfigure >/dev/null
sleep 2
gdbus call --session --dest org.kde.KWin --object-path /Scripting --method org.kde.kwin.Scripting.isScriptLoaded buddytoggle
grep -n "Buddy\|buddytoggle" ~/.config/kglobalshortcutsrc
Check
(true,) 55:BuddyScreenToggle=Meta+Shift+B,none,Buddy: toggle face / desktopPress Meta+Shift+B on bigbuddy: the face minimizes; press again: it comes back.
The touchscreen is a WingCool panel on DP-1. Its touch input is a separate USB device, and KWin must be told which
display it belongs to. On 2026-10-01 KWin had it on no display, so taps landed on the TV's coordinates.
List the touch devices and the display each one is on. The event numbers can differ on your machine. If gdbus
cannot reach the session bus over ssh, export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus first, as
the record does for its other KWin calls:
On bigbuddy:
for ev in event17 event18 event19; do
n=$(gdbus call --session --dest org.kde.KWin --object-path /org/kde/KWin/InputDevice/$ev --method org.freedesktop.DBus.Properties.Get org.kde.KWin.InputDevice name 2>/dev/null)
o=$(gdbus call --session --dest org.kde.KWin --object-path /org/kde/KWin/InputDevice/$ev --method org.freedesktop.DBus.Properties.Get org.kde.KWin.InputDevice outputName 2>/dev/null)
t=$(gdbus call --session --dest org.kde.KWin --object-path /org/kde/KWin/InputDevice/$ev --method org.freedesktop.DBus.Properties.Get org.kde.KWin.InputDevice touch 2>/dev/null)
e=$(gdbus call --session --dest org.kde.KWin --object-path /org/kde/KWin/InputDevice/$ev --method org.freedesktop.DBus.Properties.Get org.kde.KWin.InputDevice enabled 2>/dev/null)
echo "$ev name=$n touch=$t output=$o enabled=$e"
done
Output on 2026-10-01: the device with touch=(<true>,) is the one to map, and its output is empty.
event17 name=(<'WingCool Inc. TouchScreen Stylus'>,) touch=(<false>,) output=(<''>,) enabled=(<true>,)
event18 name=(<'WingCool Inc. TouchScreen'>,) touch=(<true>,) output=(<''>,) enabled=(<true>,)
event19 name=(<'WingCool Inc. TouchScreen'>,) touch=(<false>,) output=(<''>,) enabled=(<true>,)
Map it to DP-1. KDE saves the mapping in ~/.config/kcminputrc by itself:
On bigbuddy:
gdbus call --session --dest org.kde.KWin --object-path /org/kde/KWin/InputDevice/event18 --method org.freedesktop.DBus.Properties.Set org.kde.KWin.InputDevice outputName "<'DP-1'>"
gdbus call --session --dest org.kde.KWin --object-path /org/kde/KWin/InputDevice/event18 --method org.freedesktop.DBus.Properties.Get org.kde.KWin.InputDevice outputName
grep -A1 -i "wingcool" ~/.config/kcminputrc
Check
() (<'DP-1'>,) [Libinput][10182][1321][WingCool Inc. TouchScreen] OutputUuid=6d8b2ab8-1258-4d10-8c02-89a07c2bb046Tap the face: Buddy's log shows
TAP (touchscreen)and he listens, as if you had said "yo buddy". A real tap
was verified 2026-10-01 14:37.
What the face does with touch (voice/face/face3d.html, sent to the face server): a tap listens; swipe left or
right is next or previous track; swipe up or down is music volume; a long press pauses or resumes; a double tap
plays the current song's album. Only clean finger or mouse taps count (50 to 500 ms, under 20 px, primary pointer);
the panel's stylus device produced phantom taps on 2026-10-02, and every ignored touch is logged as
touch: ignored ... for diagnosis. The ROBOT button opens /robot: the robot's live camera, battery and state,
and STOP, Come here, Go home, Look around.
One display change at a time
On 2026-10-01 a run of stacked display changes on bigbuddy (moving the kiosk between displays, changing the
primary display, disabling and enabling the TV output, brightness and DDC/CI) left the touchscreen black, and
the kernel loggedHDMI FRL link training failedwhile the TV was off but its output enabled. The owner had
it all restored to defaults. Change one display setting, look at the screen, then the next
(docs/lessons.md2026-10-01).
The owner's server runs two web apps, BUDDY JUKEBOX and BUDDY VIDEO, which read his Plex library on the server
(192.168.1.122:32400). bigbuddy runs modified copies of both in containers, bound to loopback, so Buddy can
control them (owner's OK 2026-09-28: "copy and host a modified container on big buddy"). The sources are in the
repo under media/jukebox and media/video (Flask apps, python:3.12-alpine, flask==3.0.3, requests==2.32.3).
What uses them today:
voice/music.py: mpv in a transient user unit buddy-music, playing tracks the jukebox/api/browse, /api/search), into buddy_aec_sink, which forwards to the Evo 150. Thevoice/README.md says music goes straight to the Evo sink; since 2026-10-02 music.py plays intobuddy_aec_sink so the echo canceller has it as reference.media_screen.play_movie()).New idea: containers and quadlets
A container runs a program with its own files and libraries, isolated from the rest of the machine, from
an image built from aDockerfile. podman runs containers without a daemon and without root. A
quadlet is a small.containerfile in~/.config/containers/systemd/; ondaemon-reload, systemd
turns it into an ordinary user service (buddy-jukebox.service), so the container starts at boot and
restarts on failure like any other unit.
The record copied the apps with tar from a scratch copy on the laptop. macOS tar adds extended attributes Linux tar
warns about (tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'); the video
copy used --no-xattrs, which avoids that. With the repo as the source:
On your laptop:
tar cf - --no-xattrs -C ~/CCode/rosorin-pro/media jukebox video 2>/dev/null | ssh bigbuddy 'mkdir -p ~/media-apps && tar xf - -C ~/media-apps 2>/dev/null'
The live copies on bigbuddy also hold macOS ._* files from an earlier copy; they are harmless.
Both containers read PLEX_TOKEN from an environment file. The token lives in ~/media-apps/plex_token (mode 0600)
and nowhere else on bigbuddy; never put it in the repo, a unit file or a command line you paste into a chat. The
owner wrote that file from the server's ~/openwebui-stack/.env through a pipe.
Not recorded: the token copy
The exact command the owner used to fill~/media-apps/plex_tokenis not in the record. What the file must
contain is the token alone, on one line.
If it fails
The firstplex_tokenon 2026-09-28 held a${...}variable reference copied from the compose file, not the
token. This check prints nothing secret:On your laptop:
ssh bigbuddy 'grep -q "^\\\${" ~/media-apps/plex_token && echo "file holds a \${...} reference, not a token" || echo "not a reference"'
not a referenceis what you want.
Then write the two environment files from it, readable only by you. The printf reads the token from the file, so
it is never printed:
On bigbuddy:
cd ~/media-apps
umask 077; printf 'PLEX_TOKEN=%s\n' "$(tr -d ' \r\n' < plex_token)" > jukebox.env
umask 077; printf 'PLEX_TOKEN=%s\n' "$(tr -d ' \r\n' < plex_token)" > video.env
ls -la jukebox.env video.env | awk '{print $1, $5, $9}'
The apps send the token to Plex in the X-Plex-Token header, not in the URL, so error logs cannot print it
(changed 2026-09-29 after an HTTP error had logged the full URL). The video app's HLS proxy still passes it as a URL
parameter, which is never logged.
Build the images:
On bigbuddy:
cd ~/media-apps && podman build -q -t localhost/buddy-jukebox:latest jukebox && podman build -q -t localhost/buddy-video:latest video
The quadlets are in the repo, media/buddy-jukebox.container:
On bigbuddy:
# BUDDY JUKEBOX copy (widescreen TV layout, artist search, remote control for Buddy). Source: ~/media-apps/jukebox
# (copied from server ~/openwebui-stack/jukebox 2026-09-28). Local only.
[Unit]
Description=Buddy jukebox (bigbuddy copy)
[Container]
Image=localhost/buddy-jukebox:latest
ContainerName=buddy-jukebox
PublishPort=127.0.0.1:8281:80
EnvironmentFile=%h/media-apps/jukebox.env
Environment=PLEX_URL=http://192.168.1.122:32400
Environment=PLEX_MUSIC_SECTIONS=19,9
[Service]
Restart=always
[Install]
WantedBy=default.target
and media/buddy-video.container, the same shape with port 8282, video.env and the movie sections:
On bigbuddy:
# BUDDY VIDEO copy (frame off by default, per-style effect strength, remote control for Buddy).
# Source: ~/media-apps/video (copied from server ~/openwebui-stack/video 2026-09-28). Local only.
[Unit]
Description=Buddy video (bigbuddy copy)
[Container]
Image=localhost/buddy-video:latest
ContainerName=buddy-video
PublishPort=127.0.0.1:8282:80
EnvironmentFile=%h/media-apps/video.env
Environment=PLEX_URL=http://192.168.1.122:32400
Environment=PLEX_VIDEO_SECTIONS=1,2,3,20
[Service]
Restart=always
[Install]
WantedBy=default.target
PublishPort=127.0.0.1:8281:80 makes the app inside the container (port 80) reachable on bigbuddy's loopback only.
The section numbers are Plex library IDs on the server. Install and start:
On your laptop:
ssh bigbuddy 'mkdir -p ~/.config/containers/systemd' && scp -q ~/CCode/rosorin-pro/media/buddy-jukebox.container ~/CCode/rosorin-pro/media/buddy-video.container bigbuddy:.config/containers/systemd/
On bigbuddy:
systemctl --user daemon-reload && systemctl --user start buddy-jukebox.service buddy-video.service; sleep 6
systemctl --user is-active buddy-jukebox.service buddy-video.service
curl -s -m 10 http://127.0.0.1:8282/api/init; echo
for p in 8281 8282; do curl -s 127.0.0.1:$p/ | grep -c 8290/overlay; done
Check
activetwice. The video app's/api/initon 2026-09-28:{"channel_list":[{"id":"1","index":0,"title":"Family"},{"id":"2","index":1,"title":"DVD"},{"id":"3","index":2,"title":"Blu-Ray"},{"id":"20","index":3,"title":"Robotech Blu-Ray"}],"channels":4,"total_movies":891}The overlay check prints
1and1: both pages load Buddy's corner face from the face server. Quadlet units
are generated, not enabled;WantedBy=default.targetin the[Install]section starts them at boot.
After editing an app, rebuild its image and restart its unit, for example
cd ~/media-apps && podman build -t localhost/buddy-jukebox:latest jukebox && systemctl --user restart buddy-jukebox
(media/README.md).
The record differs in two places. media/README.md still describes a Firefox kiosk with PULSE_SINK=HDMI; the
kiosk is Chromium on buddy_aec_sink. The running jukebox image (built 2026-09-29) and the live
~/media-apps/jukebox/static/index.html lack the repo's album remote command and end-of-album shuffle (added
2026-10-01); building from the repo gives you the newer page, which has not run on bigbuddy.
Three small modules give Buddy his room tools.
Home Assistant (voice/home.py). Home Assistant runs on the server in Docker Desktop. Buddy calls its REST API
at http://192.168.1.122:8123, with two IPv6 addresses of the server as fallbacks: on 2026-09-28 Home Assistant's
IPv4 had dropped and only IPv6 answered, until docker restart homeassistant on the server. The token is a
long-lived access token named "buddy", created by the owner in Home Assistant, in ~/voice/ha_token (mode 0600).
Buddy only controls lights (light.* entities) and the TV (media_player.den_den_tv). He never touches
switches: the den area also holds the Evo 150's switches.
Not recorded: entering the Home Assistant token
The owner created the token and typed it at a hidden prompt on 2026-09-28. The exact command is not in the
record. The file must hold the token alone.
On bigbuddy:
cd ~/voice; V=~/voice/venv/bin/python
$V home.py list; $V home.py status den
Check
Output on 2026-09-28:areas: bedroom, den, kitchen, living room, porch; lights: Den Lights, Drums, Burgerbarn Tube1, Burgerbarn Tube2, Luna, Desk, PaulFrank, Fan2, Gollum, Record Wall, Fan1, Records, Mini The den lights are off.
The Evo 150 (voice/evo.py). The amplifier is controlled directly over its own StreamMagic HTTP API at
http://192.168.1.11, no token and no Home Assistant (owner 2026-10-01). Sound reaches it over USB from bigbuddy,
so its input must stay on USB_AUDIO. ensure_usb() switches it back when it drifts (after a power cycle it falls
back to its TV input): at start-up, before media starts and before Buddy speaks, at most once per 30 s, and never in
Apple TV mode. Volume is capped at 80 %.
The bigbuddy side of the Evo's USB sink stays at 100 %; the amplifier's own volume is the only room volume
(docs/lessons.md 2026-10-06: the sink at 55 % made voice and music "really quiet").
On bigbuddy:
cd ~/voice && venv/bin/python evo.py status && venv/bin/python -c "import evo; print('ensure_usb ->', evo.ensure_usb())"
Check
Output on 2026-10-01:The Evo 150 is on USB Audio, volume 49 percent. ensure_usb -> None
Nonemeans nothing needed changing.
Room modes (voice/modes.py, 64 lines). One voice tool, room_mode, sets the TV, the displays and the
amplifier together. The whole file:
On bigbuddy:
"""Room modes, switched by voice (owner 2026-10-01). Sound is ALWAYS the Evo 150 over USB (no HDMI audio).
game = Samsung TV on and the primary display (games/Steam open there); Buddy's face stays on the touchscreen.
buddy = Buddy-only: TV off, the touchscreen is the primary display.
appletv = Samsung on its HDMI input (Apple TV), Evo on TV ARC (the TV's sound); Buddy stops forcing the Evo to USB
until another mode is chosen (MODE_FILE).
TV power via Home Assistant (the Samsung isn't attached to bigbuddy); displays via kscreen-doctor (set only, no
queries - queries poll the touchscreen over DDC/CI)."""
import os, subprocess
MODE_FILE = os.path.expanduser('~/.cache/buddy-room-mode')
def current():
try:
return open(MODE_FILE).read().strip()
except OSError:
return 'buddy'
ENV = dict(os.environ, XDG_RUNTIME_DIR=f'/run/user/{os.getuid()}', WAYLAND_DISPLAY='wayland-0')
def _screens(*args):
# The two displays never share a position (2026-10-05): with both at 0,0 a full-screen Buddy took the big
# display's size and the 1280x720 touchscreen showed its top-left corner ("all I see is one eye"). The
# touchscreen keeps 0,0; the HDMI display sits to its right.
subprocess.run(['kscreen-doctor', *args, 'output.DP-1.position.0,0', 'output.HDMI-A-1.position.1280,0'],
env=ENV, capture_output=True, timeout=15)
def _face_on_touchscreen():
"""Buddy's window full screen on the touchscreen, whatever it was before."""
import media_screen
try:
media_screen._kwin("const o = workspace.screens.find(s => s.name === '%s');\n"
"if (w && o) { const g = o.geometry; w.minimized = false; w.fullScreen = false; w.keepAbove = false; "
"w.noBorder = false; workspace.sendClientToScreen(w, o); "
"w.frameGeometry = {x: g.x, y: g.y, width: g.width, height: g.height}; w.fullScreen = true; }"
% media_screen.FACE_OUTPUT, 'buddyplace')
except Exception:
pass
def set_mode(mode):
import evo, home
if mode in ('game', 'buddy', 'appletv'):
open(MODE_FILE, 'w').write(mode)
if mode == 'appletv':
tv = home.tv('on'); home.tv('source', 'HDMI')
amp = evo.set_source('tv arc')
return f'Apple TV mode: {tv} TV on its HDMI input. {amp} I will leave the speakers on the TV until you switch modes.'
if mode == 'game':
tv = home.tv('on')
_screens('output.HDMI-A-1.enable', 'output.HDMI-A-1.priority.1', 'output.DP-1.priority.2')
_face_on_touchscreen()
evo.ensure_usb(max_age=0)
return f'Game mode: {tv} The TV is the main screen; sound stays on the Evo over USB.'
if mode == 'buddy':
tv = home.tv('off')
_screens('output.DP-1.priority.1', 'output.HDMI-A-1.priority.2')
_face_on_touchscreen()
evo.ensure_usb(max_age=0)
return f'Buddy mode: {tv} The touchscreen is the main screen.'
return f'unknown mode {mode}'
The mode is remembered in ~/.cache/buddy-room-mode (live: appletv on 2026-10-07), which evo.ensure_usb()
reads so it leaves the amplifier on the TV's sound in Apple TV mode. Note the comment in _screens: the two
displays must never share a position.
curl -s -o /dev/null -w "face %{http_code}\n" 127.0.0.1:8290/face on bigbuddy prints face 200, and /eventsstate line.systemctl --user stop buddy-screen.service; systemctl --user start buddy-face-screen.service, the page[1280, 720, ...] on this touchscreen).systemctl --user is-active buddy-jukebox.service buddy-video.service prints active twice and the video app's/api/init lists the four channels.home.py list names your Home Assistant areas and evo.py status reports the Evo's input and volume.127.0.0.1 (repo) or 0.0.0.0 (live), knowing what/robot/do can start.Where this comes from
Repo (github.com/burgerbarn/rosorin-pro, branchrebuild, HEADbfb61d8):voice/face_server.py,
voice/media_screen.py,voice/buddy-face-screen.service,voice/kwin/buddytoggle/,voice/face/face3d.html,
voice/music.py,voice/home.py,voice/evo.py,voice/modes.py,voice/buddy_voice.py(main, tools),
voice/README.md(Face, 3D face, Boot, Touchscreen, Evo, Music sections),media/README.md,
media/buddy-jukebox.container,media/buddy-video.container,media/jukebox/,media/video/,
robot_face/face_fb.py,systemd/rosorin-face.service,scripts/ops/robot_mode.sh,docs/decisions.md
2026-09-30 (Chromium) and 2026-10-07 (robot headless),docs/lessons.md2026-10-01 (touchscreen black) and
2026-10-06 (Evo volume stages).survey_bigbuddy.md(component map, phases 1-3, live differences: autologin,
face-screen enabled, face server bind). Live reads 2026-10-07 (read-only):face_server.pyline 286 and its
backup, listening ports, unit states,kwinrc,kglobalshortcutsrc,kcminputrc, env file key names and
modes (no values), quadlet files, image dates, package versions. Command logs: quadlets 2026-09-28 20:25 and
20:53 UTC, Plex token check 20:23 UTC, Home Assistant test 20:35 UTC; face server checks 2026-09-29 04:46 UTC;
KWin script 2026-09-29 05:36 UTC; face-screen unit 2026-10-01; touch mapping 2026-10-01 21:37 UTC; Evo test
2026-10-01 22:16 UTC; full-screen fix and boot-path test 2026-10-07 17:21-17:24 UTC.