How AutoYou works.

Architecture

How the pieces connect

AutoYou has three layers: the mobile app, autoyou_lib (the Python server on your computer), and an optional AI pipeline. The app and server pair once via a messaging platform, then talk directly P2P over WebRTC for everything after.

AutoYou App (iOS / Android)
    │
    │  ① OTP / AutoPair  ──→  tunnelmole URL  -or-  AutoYou Cloud relay
    │  ② WebRTC / SRTP   ──→  direct P2P (after signaling)
    ▼
autoyou_lib  (Python, your machine)
    │  ③ DataChannel (SCTP) ── chat messages
    │                       ── HTTP proxy requests (browser)
    │                       ── file transfers, ping/pong
    │  ④ SRTP audio track  ── TTS audio to app
    │  ⑤ AudioTrackSink    ── Whisper STT from mic
    │  ⑥ SSE /api/v1/events
    ▼
OpenClaw autoyou plugin  (Node.js, optional)
    ├── routes messages → OpenClaw AI pipeline
    ├── fallback        → autoyou_lib /v1/chat/completions
    └── registers       → autoyou/default model provider

Prerequisites

What you need

The core route requires only Python and the AutoYou mobile app. The OpenClaw integration additionally needs Node.js.

Installation

Install and start autoyou_lib

autoyou_lib is the Python server that runs on your computer. It manages WebRTC signaling, the DataChannel, audio processing, and the local REST API.

pip install fastapi uvicorn aiohttp pydantic cryptography aiortc av

pip install openai-whisper sounddevice numpy scipy

pip install pyotp

# Open mode (default)
python -m autoyou_lib

# Secure mode (AES-256-GCM encrypted AutoPair)
python -m autoyou_lib --security-mode secure

# Secure Professional (TOTP 2FA)
python -m autoyou_lib --security-mode secure-pro --totp-secret BASE32SECRET

# With OpenClaw AI pipeline
python -m autoyou_lib --openclaw-api http://127.0.0.1:8856 \
                       --openclaw-token YOUR_TOKEN

First-time setup

OTP pairing — how it works

The first time you connect, AutoYou verifies your phone using a one-time password sent through a messaging platform you already trust. This is the only step that requires manual action — future reconnections use AutoPair automatically.

1

Get a public URL for your server

Send /pair to your bot on Telegram, WhatsApp, or Signal. The server starts tunnelmole, gets a public HTTPS URL, and replies with /otp <code> for you to forward to the AutoYou app.

# You send to your bot:
/pair

# Bot responds:
/otp 847291   ← copy this into the AutoYou app

2

Enter the OTP in the AutoYou app

Open the AutoYou app, tap Connect, enter the OTP code. The app sends it to the server's /auth endpoint via the tunnelmole URL.

3

Server verifies, WebRTC begins

The server verifies the OTP using SHA-256. On success it creates a WebRTC offer (SDP) and returns it to the app. In Secure Professional mode, the TOTP secret is returned encrypted inside the response.

4

ICE negotiation + direct link opens

The app creates a WebRTC answer, posts it back to /signal, and ICE candidate exchange completes the handshake. The DataChannel (SCTP) and audio track (SRTP) open. Tunnelmole is no longer involved — the connection is direct P2P.

Automatic reconnect

AutoPair — no codes after the first time

Once you've paired once, AutoPair handles every subsequent reconnection automatically. No more messaging bots, no more copy-pasting codes.

Security

Three security modes

Configure the server with --security-mode. The app adapts its pairing flow automatically based on what the server advertises.

Browser proxy

Access your computer's web from your phone

The DataChannel HTTP proxy tunnels HTTP requests from the app's Browser tab through the SCTP DataChannel to your computer, and forwards the response back. No ports need to be opened on your firewall.

1

Enable the proxy on the server

python -m autoyou_lib --proxy-enabled --proxy-target http://127.0.0.1:8856

--proxy-target is the local address to proxy to. Default is port 8856 (the OpenClaw admin interface).

2

Open the Browser tab in the app

The app sends HTTP_REQUEST messages over the DataChannel. The server proxies them to proxy_target and returns HTTP_RESPONSE messages. SSE streaming and WebSocket upgrade are also supported.

Voice calls

AI voice calls powered by your machine

AutoYou's voice path uses the WebRTC SRTP audio track. Your computer transcribes speech with Whisper, routes the text through your AI, and streams TTS audio back. No cloud STT subscription needed.

1

Install voice dependencies

pip install openai-whisper sounddevice numpy scipy

2

Configure STT / TTS in settings

Set the Whisper model size and TTS provider in the local admin dashboard at http://localhost:8099/admin or via server startup flags. Smaller Whisper models (tiny, base) are faster; larger ones (medium, large) are more accurate.

python -m autoyou_lib --whisper-model base

3

Tap the voice button in the app

Once the SRTP audio track is live, tap the mic icon in the AutoYou chat or voice tab. Audio flows to your machine, is transcribed, routed to your AI, and the TTS response streams back.

Paid tier

AutoYou Cloud pairing — no tunnelmole required

Paid AutoYou subscribers can enroll their server with AutoYou Cloud. The cloud acts as a signaling relay — routing AutoPair offers from the app to the server and answers back — without tunnelmole. Your actual data still flows P2P.

/autoyou-enroll https://cloud.autoyou.me YOUR_TOKEN

{
  "cloudApiUrl":
    "https://cloud.autoyou.me",
  "cloudDeviceToken":
    "YOUR_TOKEN"
}

python -m autoyou_lib \
  --cloud-api-url https://... \
  --cloud-device-token TOKEN

Cloud pairing protocol:

① Server  →  POST /api/v1/servers/register
             {device_token, server_name}
          ←  {server_id}

② Server  →  GET /api/v1/servers/{id}/inbox?timeout=30
          ←  {messages: [{id, type:"autopair_offer", payload}]}

③ Server  →  POST /api/v1/servers/{id}/messages/{msg_id}/reply
             {type:"autopair_answer", payload}

④ Cloud relays answer → App → ICE completes → P2P link opens
   (Cloud is no longer involved after this point)

Network considerations

Firewalls, tunnels, and fallback modes

During /pair, your computer creates a temporary public HTTPS URL via Tunnelmole so your phone can find it. This tunnel is only active for a few seconds during the initial handshake, and it requires a valid one-time code to accept any connection. Once the direct WebRTC link is established, the tunnel shuts down and your computer is no longer reachable from the internet.

What if my network blocks the tunnel?

Some corporate firewalls, restrictive ISPs, or network security policies may block outbound connections to the public reverse proxy. This only affects the first-time /pair — not ongoing use. If blocked:

• AutoPair (/autopair) exchanges encrypted WebRTC offers and answers through your messaging app. No public URL is needed. If you've paired once from any network, AutoPair will reconnect you from anywhere.
• Cloud Pair (paid) uses AutoYou's relay server for the handshake, completely eliminating the need for any tunnel on your computer. Only the encrypted handshake passes through the cloud — once connected, all traffic flows directly P2P.

Advanced networking

ICE, STUN, and TURN — when direct connections fail

WebRTC uses ICE (Interactive Connectivity Establishment) to find the best path between your phone and computer. On most home and office networks this works automatically. But some complex network environments — symmetric NATs, double NATs, carrier-grade NATs, or strict enterprise firewalls — can prevent ICE from completing. When this happens, neither /pair, AutoPair, nor Cloud Pair will establish a direct connection.

STUN servers

STUN servers help your devices discover their public IP addresses and port mappings. AutoYou uses Google's public STUN servers by default, which work for most networks. If you're in a region where these are blocked or slow, you can configure locally-hosted STUN servers for better performance and reliability.

TURN servers

When a direct peer-to-peer path is impossible (symmetric NAT, double NAT), TURN servers relay all traffic between your devices. TURN requires credentials and uses more bandwidth, but guarantees connectivity on even the most restrictive networks.

How to configure

You can obtain STUN/TURN credentials from providers like Metered (metered.ca/stun-turn) or host your own with coturn. Then paste the ICE server configuration into either:

• AutoYou app: Settings → ICE Server Bundle
• AutoYou-Computer: Admin UI at localhost:8001 → ICE Servers

The configuration is a JSON array:

[
  {"urls": "stun:stun.your-region.example.com:3478"},
  {
    "urls": "turn:turn.your-region.example.com:3478",
    "username": "your-username",
    "credential": "your-password"
  }
]

For a comprehensive guide on WebRTC connectivity, choosing STUN/TURN providers, and diagnosing ICE failures, search for "WebRTC TURN server guide" — the WebRTC.org documentation and Metered's guides are excellent starting points.

Troubleshooting

Common issues and fixes