Sharkord
Plugins

Voice

Publish external audio and video into voice channels, and receive what users are sending.

The voice API is the low-level integration point for plugins that touch media: music bots, IPTV bridges, soundboards, recorders, transcribers. It hands you the raw mediasoup router for a channel, lets you register your own producers as a stream users can see and hear, and lets you consume what people in the channel are sending.

This assumes you know mediasoup. If you do not, read its documentation first; nothing here replaces it.

The API

MethodReturns
ctx.voice.getRouter(channelId)The mediasoup Router for that voice channel
ctx.voice.createWebRtcTransport(channelId)A mediasoup WebRtcTransport on the server's WebRTC port
ctx.voice.getListenInfo(){ ip, announcedAddress }
ctx.voice.getState(channelId)Who is connected, what they muted, and plugin streams
ctx.voice.getProducers(channelId)Every live producer in the channel
ctx.voice.createStream(options)A handle to the stream you created
ctx.voice.consume(options)A handle to a user's media, as RTP packets
ctx.voice.react(channelId, userId, emoji)Floats an emoji on a user's card
ctx.voice.move(userId, channelId)Sends a user to another voice channel
ctx.voice.disconnect(userId)Takes a user out of voice
ctx.voice.setServerMute(userId, muted)Mutes a user for everyone, or lifts it

Everything but getListenInfo, move, disconnect and setServerMute throws if the channel has no voice runtime. A runtime exists only while someone is in the channel, so wait for the voice:runtime_initialized event rather than assuming one is there.

getListenInfo is synchronous and returns the address Sharkord is listening on and the address it advertises to clients (webRtc.announcedAddress, when set). Use it when building your own transports.

Creating a Stream

The flow is: get the router, create your transports and producers, then register them.

import type { PluginContext } from "@sharkord/plugin-sdk";

const VIDEO_SSRC = 11111111;
const AUDIO_SSRC = 22222222;

const onLoad = async (ctx: PluginContext) => {
  ctx.events.on("voice:runtime_initialized", async ({ channelId }) => {
    const router = ctx.voice.getRouter(channelId);
    const { ip, announcedAddress } = ctx.voice.getListenInfo();

    const transport = await router.createPlainTransport({
      listenInfo: { ip, protocol: "udp", announcedAddress },
      rtcpMux: true,
      comedia: true,
    });

    const audioProducer = await transport.produce({
      kind: "audio",
      rtpParameters: {
        codecs: [
          {
            mimeType: "audio/opus",
            payloadType: 111,
            clockRate: 48000,
            channels: 2,
            parameters: {},
            rtcpFeedback: [],
          },
        ],
        encodings: [{ ssrc: AUDIO_SSRC }],
      },
    });

    const stream = ctx.voice.createStream({
      channelId,
      title: "Studio Feed",
      key: "studio-feed",
      producers: { audio: audioProducer },
    });

    stream.update({ title: "Studio Feed Live" });
    // stream.remove() when you are done
  });
};

Send RTP to the transport's address and port, and it appears in the channel. Video works the same way, with a video producer on its own transport; the SDK re-exports the mediasoup types you need (Router, PlainTransport, Producer, RtpParameters, and friends).

createStream Options

OptionRequiredDescription
channelIdyesThe voice channel to publish into
titleyesShown to users
keyyesYour identifier for the stream
producersyes{ audio?, video? } mediasoup producers
avatarUrlnoAvatar shown next to the stream
bannerUrlnoBanner image for the stream
ownerUserIdnoThe user streaming through your plugin. Their client starts the stream muted so they do not hear themselves back, and they can raise it with the normal volume control
videoLayersnoSimulcast layer labels, only used when simulcast is on in the server settings

It returns:

{
  streamId: number;
  update(options): void;  // title, avatarUrl, bannerUrl, producers, videoLayers
  remove(): void;
}

Receiving WebRTC Media

ctx.voice.createWebRtcTransport(channelId) creates a mediasoup WebRtcTransport on the channel's router that shares the server's WebRTC port, the same one browsers connect to. That is what a WHIP ingest needs: OBS sends WebRTC to your transport and no extra ports have to be opened.

const transport = await ctx.voice.createWebRtcTransport(channelId);

// answer the client's SDP offer with these, then call transport.connect()
// with the DTLS parameters from the offer
const { iceParameters, iceCandidates, dtlsParameters } = transport;

const audio = await transport.produce({ kind: "audio", rtpParameters });

ctx.voice.createStream({
  channelId,
  title: "OBS",
  key: "obs",
  ownerUserId: userId,
  producers: { audio },
});

The transport is configured the way Sharkord configures its own: UDP and TCP, and the server's webRtc.maxBitrate. It throws when the channel has no voice runtime.

Pass the streaming user as ownerUserId when you register the stream, so they do not hear an echo of their own audio.

Reading the Channel

getState is the snapshot of who is in the channel and what they have turned on:

const { users, externalStreams } = ctx.voice.getState(channelId);

for (const { userId, state } of users) {
  // state: { micMuted, soundMuted, webcamEnabled, sharingScreen, serverMuted }
}

externalStreams is keyed by stream id and holds every plugin stream playing in the channel, yours and other plugins'.

getProducers is the list of live media in the channel:

const producers = ctx.voice.getProducers(channelId);
// [{ userId, kind, producerId, paused }]

kind is a StreamKind: AUDIO, VIDEO, SCREEN or SCREEN_AUDIO. paused is true while the user has that track muted or stopped, and the producer stays in the list either way.

StreamKind is not re-exported yet

The SDK's own entry point does not export the StreamKind enum, so importing it from @sharkord/plugin-sdk fails today. Until it does, take it from @sharkord/shared.

Consuming a User's Media

ctx.voice.consume hands you a user's media as RTP packets, inside the Sharkord process. No ports are opened and no external program is involved.

import { StreamKind, type PluginContext } from "@sharkord/plugin-sdk";

const onLoad = async (ctx: PluginContext) => {
  ctx.events.on("voice:producer_added", async ({ channelId, userId, kind }) => {
    if (kind !== StreamKind.AUDIO) return;

    const consumer = await ctx.voice.consume({
      channelId,
      userId,
      kind,
      onRtp: (packet) => buffer.push(packet),
    });

    // consumer.close() when you are done
  });
};
OptionDescription
channelIdThe voice channel
userIdWhose media to consume
kindStreamKind.AUDIO, VIDEO, SCREEN or SCREEN_AUDIO
onRtpCalled with one RTP packet, header included

The handle carries producerId, the rtpParameters describing what the packets are, requestKeyFrame() for video, and close().

consume throws when that user has no producer of that kind in the channel, so a plugin that starts mid-call has to read getProducers as well as listening for voice:producer_added, or it only hears whoever unmutes after it started.

What arrives is what the browser sent

Audio is Opus at 48 kHz, and decoding it is yours to do. When what you want is a file, do not decode packets yourself: create a plain transport from getRouter() and consume the same producerId into ffmpeg, which gets you decoding, resampling and muxing for free.

onRtp is called often, roughly every 20 ms per speaker for audio. Keep it to buffering and do the work elsewhere.

Consuming is recording

A plugin that consumes is listening to people in a voice channel, and nothing in the interface tells them so. Sharkord writes a line to the plugin's log when a consumer starts and stops, which is what an admin can find later. If you ship this, say so.

Reactions

ctx.voice.react floats an emoji on a user's card, the same animation as a reaction they sent themselves:

await ctx.voice.react(channelId, userId, "🔊");

It takes a unicode emoji, a shortcode or the name of a custom emoji, and throws when the user is not in the channel or the emoji is unknown. Nothing marks it as coming from a plugin: to everyone watching, that user reacted. For anything with text, such as the name of the sound playing, use the VOICE_USER_CARD slot instead.

Moderation

Three calls do what a moderator does by hand, for join-to-create channels, AFK movers and music bots.

await ctx.voice.move(userId, channelId);
await ctx.voice.disconnect(userId);
await ctx.voice.setServerMute(userId, true);

move sends a user to another voice channel. Their client makes the switch, so it lands a moment later, and you see it as user:left_voice then user:joined_voice. The user needs neither the channel's JOIN permission nor sight of it: a move can pull someone into a private channel, which is then revealed to them. They do still need the server-wide permission to use voice. It throws when the destination is not a voice channel, or when the user is not in voice or already in that channel.

disconnect takes a user out of whatever voice channel they are in, and throws when they are not in one.

Neither reaches into a direct message call. A user in one is treated as not being in voice at all, so a plugin cannot tell that the call exists.

setServerMute mutes a user for everyone, or lifts it with false:

  • Their audio is stopped on the server, not left to their client, so a modified client cannot get around it.
  • They cannot unmute themselves while it lasts; trying shows them that a moderator muted them.
  • It covers every voice channel, so leaving, rejoining or switching channels does not shake it off, and it can be set while the user is not in voice.
  • It lasts until lifted or the server restarts.
  • Lifting it leaves them muted. They unmute themselves when they are ready.

state.serverMuted in getState and in the voice:state_updated event says whether a user is muted this way, as opposed to having muted themselves.

None of these check who asked. A plugin acts as the server, so gate the command or action that calls them yourself, for example with ctx.permissions.userCan(invoker.userId, Permission.MOVE_MEMBERS).

Cleaning Up

Keep every handle you create and call remove() on streams in onUnload. ctx.voice is available on the unload context for exactly this. Streams left behind stay visible to users until the channel's runtime closes.

Consumers are the exception: they close themselves when the producer ends and when the plugin unloads. Closing one yourself is how you stop early.

Transports from createWebRtcTransport are yours to close, but whatever is still open when the plugin unloads is closed for you, and they close with the channel's runtime.

On this page