PhoenixDevelopers

Jibo APIs for skills

On this page

Developer guide · App context · Recipes

Use the initialized context.services.jibo and context.services.Be; do not bundle a second SDK or start global services yourself. This guide covers the skill-facing surfaces and their ownership rules. For the original SDK's class, enum and asset-schema details, use the SDK reference with the installed software's API. An old SDK entry is not a promise that an optional service, remote provider or asset is present on every Jibo.

Runtime and resource budget

  • Robot service code targets Node 6.9.2; Electron 1.4.3 embeds Node 6.5.0 and Chrome 53. Write skills for the stricter embedded Node.
  • Use CommonJS require, var/ES2015 syntax and Promises with .then(). No async/await, optional chaining, object spread, ESM imports, fetch, AbortController, fs.promises or modern CSS/browser API assumptions.
  • Run the kit itself with Node 20+ on your computer. Its default syntax check uses the host parser. Add --syntax-node "$NODE_6_5" for the exact trusted local Node 6.5.0 parser; Jibo still parses independently.
  • Budget for 2 GB RAM and no swap, shared with Jibo's services. These are whole-robot resources, not an app allowance. Avoid large decoded images, unbounded caches, video buffers, huge JSON and CPU-heavy synchronous loops.
  • Startup and stop are bounded by the host. Offload expensive preparation to your computer; ship small assets and JavaScript dependencies without install scripts or native binary modules. Never block the main renderer/event loop.

Surface map

SurfaceWhat a skill can doOwnership / limit
Be.BeSkill, Be.SkillSwitchData, be.skills, be.redirect, be.exitParticipate in BE's skill switching and screen lifecycleRegister only your unique skill name, forward exit/redirect events, remove your exact registration
jibo.face.viewsCreate native views, Labels, Buttons, transitions and gesturesOwn one view, guard asynchronous callbacks, wait for transitions and verify removal
jibo.face.tween, jibo.rendering, PIXIRender and animate native componentsChrome 53 renderer; free owned display objects, textures and tweens
jibo.embodied.speech, jibo.ttsSpeech with coordinated motion/ESML, or lower-level TTSOne speech owner; retain/cancel only owned work
jibo.jetstream, jibo.embodied.listen, jibo.mim, jibo.bt, jibo.flowLocal listening turns, dialog behavior, MIM/behavior/flow assetsLocal request handles and typed events; no implicit local voice alias registration
jibo.loader and loaded Sound objectsLoad packaged sounds/images/data and cache assetsBound caches; destroy only your own sound/assets
jibo.animDB, jibo.expressionCatalog animations, head/body movement, attention, LED colorUse installed catalog names or your own assets; retain handles, respect focus and faults
jibo.system, jibo.globalEventsTouch, power/temperature telemetry, volume/backlight and shared eventsSDK typed events use .on(fn) / .off(fn); do not suppress global STOP
jibo.mediaTake photos, display photo textures or viewfinderExplicit owner intent and declared camera behavior; household media is private
jibo.lps, jibo.kbLocal perception and knowledge-base dataCheck freshness and availability; do not log identities or silently persist household information
Node http, https, net, fs, path, timersWeb APIs, your own service, settings, bounded workFull access is not isolation; close all handles; preserve TLS verification
jibo.log, jibo.performanceSDK diagnosticsKeep diagnostics bounded and free of private data

Other shared runtime namespaces include services/session/remote integration, Wi-Fi, scheduler, system manager, analytics, secure transfer, loader/plugin infrastructure, interaction memory (im), action/emotion, location and versions. They serve BE and its built-in behavior; they are not extra app-context services or permission grants. Do not reinitialize or shut them down, rewrite identity, change the owner's Wi-Fi, schedule arbitrary system work, or register colliding plugins. The historical internal ics service is not a skill API. Use the exposed perception/knowledge surfaces instead of assuming an ics object works.

BE skills and focus

The hello, touch and voice starters subclass Be.BeSkill with an asset pack name and root path. BE owns switching between skills; the native host owns installation. create(context) is not itself a screen-open event.

var Be = context.services.Be, be = context.services.be;
// After acquiring attention/speech and registering your skill:
be.redirect(new Be.SkillSwitchData(skill, { source: 'native_app' }));

Wire the skill's exit, redirect and refresh events to BE during prepare; the public entry.js starters show the exact forwarding shape. skill.exit() asks BE to leave. Implement both close(done) and destroy(done): the base class does not remove your custom view. Never delete another skill's registry entry; test identity before deleting your own.

Native views and components

The display canvas is 1280×720. Put the main content and touch controls in the central safe area; use large labels and at least 64-pixel-high touch targets. Use text, abstract skill glyphs or licensed real photos; never draw Jibo.

var views = jibo.face.views;
var view = views.createView('View', {
    viewConfig: { type: 'View', id: 'counter', category: 'gui',
        ignoreSwipeDown: true, transitionStageOnly: true },
    componentConfigs: [
        { type: 'Label', id: 'title', text: 'Counter',
          position: { x: 640, y: 240 }, targetAnchor: { x: 0.5, y: 0.5 },
          bounds: { width: 420, height: 70 },
          style: { fontFamily: 'Proxima Nova Soft', fontSize: 48,
              fill: '#FFFFFF', align: 'center' } },
        { type: 'Button', id: 'count', assets: [],
          position: { x: 440, y: 320 },
          hitArea: { x: 0, y: 0, width: 400, height: 100 },
          action: { type: views.ActionData.CALLBACK,
              data: { callback: function () { /* guard ownership, then act */ } } } }
    ]
});

viewConfig.id identifies the view; category: 'gui' creates a GUI screen. ignoreSwipeDown means your app handles leaving explicitly. Labels expose text, style, position, targetAnchor and bounds. A Button's hit rectangle is local to its position. A hit target does not automatically paint a visible button: add its background to the component's display after load. Keep text and hit-area coordinates aligned.

Open with views.changeView(options, complete, failure, loaded):

view.viewStateStacked = true;
views.changeView({ addView: view, transitionOpen: views.TRANSITION.IN,
    transitionClose: views.TRANSITION.OUT }, onComplete, onFailure, onLoaded);

Create PIXI decoration in onLoaded; enable interaction and speak after onComplete. Update text with view.getComponentById('title').text. A callback should confirm the same generation/view, views.currentView === view, and !view.inputLocked. An old callback must not draw over a replacement screen.

For swipe-down, view.addAction(views.ActionData.CALLBACK, {callback: leave}, true, false, views.GESTURE.SWIPE_DOWN) registers the exit gesture. The starters also exit on head touch and a new Hey Jibo. During close, listen to views.events.process while a transition is pending, then remove only your own current view with {remove:true, transitionClose:views.TRANSITION.NONE}. Verify view.state === 'destroyed'; a covered view is not proof of cleanup. Never remove whichever view happens to be current if it is not yours.

Embodied speech and ESML

jibo.embodied.speech.speak(text, options, autoRuleConfig) returns a Promise for a request status. The installed statuses include SUCCEEDED and STOPPED; Promise fulfillment alone does not mean the utterance completed normally. The namespace also exposes RequestStatus, isActive(), waitForAnimDispatch() and setPaths(assetPack); changing asset paths affects the shared speech engine. stop() is global to embodied speech, so call it only after confirming the active/pending session belongs to your skill. The templates include that ownership check; do not copy only their speak() line and discard cleanup.

// Fixed author text, while this app owns speech and no other speaker is active.
jibo.embodied.speech.speak('Hello! <break size="0.2"/> Ready?', {
    disableAutoRules: true, dispatchComplete: 'TTS'
});

The installed option names are case-sensitive properties:

OptionType / effect
disableAutoRulesBoolean; disable automatic embodied speech rules
dispatchCompleteString TTS or ALL; choose which completion boundary resolves dispatch
enableLayerMergingBoolean; allow merging animation layers
frameBufferNonnegative number, timing buffer in seconds
followedByListenBoolean; selects TTS completion and trims trailing animation timing for an ensuing listen
assetRelativeEndNumber; relative asset end timing in seconds
debugModeBoolean; SDK debugging

The optional auto-rule configuration controls punctuation, hot words, voice, beat and the other installed rule categories; use defaults unless your authored speech needs a specific change. followedByListen does not start a local microphone request; call the listening API once after the owned speech result.

ESML combines text with speech markup and catalog-backed expressive assets. The fixed <break size="0.2"/> example inserts a pause; installed speech markup also handles pronunciation, pitch/duration shaping and asset references. The embodied SDK reference describes the full authored markup and asset grammar. Resolve animation/sound identifiers against the installed catalog, not a guessed expression name. A web SSML tag is not automatically a supported ESML tag. Do not add a fabricated <speak> wrapper or arbitrary HTML.

Treat remote/user text as text, never trusted ESML. Escape XML-sensitive characters before insertion, or use a plain-text TTS path for untrusted content. Do not put transcripts, tokens or URLs with secrets into speech markup. The low-level jibo.tts service is shared with embodied speech; bypassing it does not grant independent speaker ownership.

Local listening and voice

For a one-reply interaction, jibo.jetstream.startLocalTurn(options) resolves to a LocalTurnRequest. Its .promise resolves to the final turn result. Retain the request through cancellation/completion. Read text only when result.status === 'SUCCEEDED', then result.result.asr.text. Timeout, cancellation and failure can contain partial data; discard it.

Local-turn optionMeaning
nluRulesRequired list of configured NLU contexts; [] for a text-focused reply
ignoreHJWhether Hey Jibo can interrupt the turn; preserve normal interruption by default
rejectIfBusyReject instead of interfering with an existing turn
ignoreGlobalRulesKeep a reply within your interaction rather than route a new global skill
hintPhrases, earlyEOSRecognition hints / phrases that can end capture; hints are not strict allowed-word validation
sosTimeout, maxSpeechTimeoutSeconds to start speaking / maximum utterance duration; -1 or omitted uses service defaults
languageService-supported language label
meta, logCaller metadata; do not include household/private data
clientASR, clientNLUInjected input for menu/testing; not microphone evidence
suppressedEventsEvent suppression list; leave unset for ordinary skills
jibo.jetstream.startLocalTurn({ nluRules: [], hintPhrases: ['hello'],
    ignoreGlobalRules: true, rejectIfBusy: true, sosTimeout: 6,
    maxSpeechTimeout: 8 }).then(function (request) {
    // Save request immediately. Cancel this exact request during close.
    return request.promise;
}).then(function (result) {
    if (result.status !== 'SUCCEEDED') return;
    var asr = result.result && result.result.asr;
    if (asr && typeof asr.text === 'string') { /* validate the word locally */ }
});

A request has id, status, typed events and error, .promise, .cancel() and .update(asrOrNlu, meta). cancel() resolves false if it could not cancel. When canceling an active request, treat false as unconfirmed release: the SDK can force-end the local promise after a transport failure without the service acknowledging cancellation. A terminal local status alone is not proof of release. Request status uses ACTIVE, CANCELED, FINISHED, ERROR (one L in CANCELED). Turn-result status has its own enum. Update is injected input, not an ASR test.

request.events.on(fn) / .off(fn) carry TURN_STARTED, SOS, EOS and other service events with request/transaction IDs. Global jibo.jetstream.events.localTurnStarted, sos, eos, hjHeard, localTurnResult and errors are shared. Subscribe before opening a turn if you need its first event: localTurnStarted can precede the request acknowledgement. Attribute all shared events to your request and remove listeners on terminal paths.

jibo.embodied.listen provides active/ambient listening presentation, including getActiveMode(), getAmbientMode(), setAmbientMode(mode), enterActiveMode(mode,options), exitActiveMode(fastForward), waitForIdle(fastForward) and disableOnce(). The shared dispose() and demoEngaged(turnOn) functions are host/presentation tools, not ordinary per-skill capture operations. It follows Jetstream's capture lifecycle; it is not the ASR request itself. Do not repeatedly open listening or compete with its eye/LED/motion presentation. The voice template retains late request creation, invalidates stale replies and cancels only its owned turn.

MIMs, behavior trees and flows

The initialized runtime also exposes jibo.mim, jibo.bt and jibo.flow for packaged MIM dialog, behavior trees and flow assets. Those assets resolve against the installed SDK/asset packs; they are not JavaScript-only local launch declarations or an automatic way to add arbitrary NLU grammars. Keep asset-pack names unique, include your own assets in skill.json.files, and stop owned behavior when the skill closes. Use direct local turns for the small voice starter; use the SDK reference for the complete MIM/flow schema when authoring a larger dialog.

Sound and asset loading

Use the initialized loader to load a bundled sound, then retain its Sound object and playback instance. The installed Sound has play, pause, resume, stop, destroy, volume, panning and loop; SoundInstance is the individual playback. Load through jibo.loader; there is no per-Sound load() or muted property in this implementation. Use the owned Sound's volume for per-sound gain, not the shared SoundContext's mute. Read the callback shape instead of assuming playback returns a Promise.

var path = require('path');
jibo.loader.load({ id: context.appId + '-chime', type: 'sound',
    src: path.join(context.appRoot, 'chime.wav') },
    function (error, sound) {
        if (error) return;
        // Save sound; on close stop/destroy only this owned object.
        sound.play(function () { /* playback complete */ });
    });

List chime.wav in your packed files. Use a small format the original renderer can decode, such as a short WAV. Do not assume codecs from a modern desktop. A sound completing is different from an utterance completing. Do not raise the robot's master volume just to make your app louder. Asset caching is shared: use an app-specific cache name and release only your own cache. Always provide an app-scoped sound id: without it the loader uses the filename stem, and loading another chime.wav can replace another app's sound. Do not overlap loads/reuse that ID while an earlier owned load is still outstanding.

Animation, motion, attention and LEDs

jibo.animDB.getAnimByName(name) looks up the installed animation catalog case-insensitively and returns undefined when no name matches. Treat that missing result as an error, not permission to make up a name. getAnimationNames() and getAnimationCategories() list catalog entries; search(term) and query(query) return structured search/query results. These are the installed API names, not the older collect / getAnimationInfo names found in historical references. jibo.expression is the native motion/expression surface; there is no context.motion.move() convenience API.

Expression APIResult / responsibility
createAnimation(options)Promise of AnimationInstance; use an app-specific cacheName to allow later cache release
createAndPlayAnimation(options, requestor)Promise of AnimationInstance; attach terminal event listeners immediately when resolved
destroyCaches(nameOrNames)Destroy only your own animation caches
acquireTarget({position:{x,y,z}})Promise of AcquireHandle; position is in meters, not degrees or normalized screen coordinates
pushAttentionMode(mode)Promise of AttentionHandle; call its release() when done
getAttentionMode(), setAttentionMode(mode)Read/set global base mode; prefer a releasable pushed mode
setLEDColor([r,g,b])Promise; components are in 0–1, not 0–255
awaitFace(options)Promise of cancelable face-wait handle, with its own result Promise
centerRobot(options), cleanup(options)Global-capable movement/cleanup; scope DOFs/owners and avoid disturbing another skill
blink()Procedural blink request; not screen art
indexRobot(), setSkillRoot(root), doCenterRobotOnDisconnect(allow)Global robot/host operations; not ordinary per-interaction setup/teardown
events.dofs, events.kinematics, dofs, featuresTyped motion updates, DOF sets and latest kinematic features

Animation options include path (absolute project root), src (absolute .anim asset), optional raw data, cacheName, speed, loops, and dofs. An AnimationInstance has .play(requestor), .stop() and typed events including stopped/cancelled/rejected. Use a bounded animation, retain the instance and await its stop on close. Do not loop forever without a retained stop path.

var modeHandle;
jibo.expression.pushAttentionMode(jibo.expression.AttentionMode.ENGAGED)
    .then(function (handle) { modeHandle = handle; });
// During close, including a late handle returned after close:
// await/chain modeHandle.release(); do not merely drop the reference.

Acquire the app-context attention lease as well; it coordinates apps but is not this SDK mode handle. Respect touch/STOP, range limits and axis-fault events. Do not bypass safety or repeatedly reissue a command after a fault. LED writes are global and can conflict with listening cues; use them briefly in an owned interaction and do not overwrite another activity's final state during cleanup.

Touch, volume and system information

SDK typed Events take one function, not Node's (eventName, function) signature. Available system events include touchChanged, touchOn, touchOff, inputEnergy, axisFaultOn, axisFaultOff, hatchOpen, hatchClose, pluggedIn and unplugged. touchOn is a local notification, not the global STOP handler. Observe it to leave your screen; do not override STOP. padState / getTouchState() describe six head pads.

System telemetry includes pluggedIn, batteryCharging, batteryChargeRate, inputEnergy, battery level/temperature, CPU/main-board temperature, system voltage and callback-based fan-speed/mode reads. Keep high-rate input-energy or motion callbacks lightweight; do not log every sample.

getMasterVolume(callback) and setMasterVolume(value, callback) use 0–1. getBacklight / setBacklight use the same normalized brightness range. The separate BE volume plugin uses a 1–10 user volume scale; never mix the units. Changing volume/backlight is a global owner setting, not a per-sound gain.

jibo.system.getMasterVolume(function (error, level) {
    if (!error) { /* level is normalized 0–1; show it without changing it */ }
});

Camera and photos

The initialized jibo.media wrapper offers CameraID, PhotoType, ThumbnailType, takePhoto(params, callback), photo URL/preview and texture loading, plus viewfinder support. Its service layer and the media-manager storage/upload APIs are shared; do not turn a simple capture into silent publication or household photo enumeration.

// Only after an explicit camera action by the owner; don't auto-run on startup.
jibo.media.takePhoto({ camera: jibo.media.CameraID.RIGHT,
    photoType: jibo.media.PhotoType.SNAP }, function (error, photo) {
    if (error) return;
    // Use only for the requested action. Do not log photo bytes or private URLs.
});

This wrapper always requests an unstored capture; it does not read a store option. Its callback receives {id, url}, with a preview URL, not image bytes. Persisting a photo is a separate storePhoto operation and needs explicit owner intent. Do not confuse wrapper options with lower-level media-service arguments.

Capture is asynchronous and cannot be undone by invalidating a callback. Guard the consumer with a generation check, discard late output, and close only your viewfinder when leaving. A declared camera label is disclosure, not a sandbox-enforced permission boundary.

Perception and knowledge

jibo.lps exposes local perception; jibo.expression.features gives latest kinematics. Interpret timestamps, coordinate frames and unknown values instead of treating a cached face/person detection as a current observation. jibo.kb is the shared knowledge-base client; identity, media and settings data belong to the owner. Prefer your app's own bounded data under dataRoot. Do not register colliding global models or use KB access as permission to collect, transmit or persist household information.

Network, filesystem and timers

Use Node https for web APIs; there is no global fetch on this runtime. Set request/response limits, check HTTP status, handle premature close, reject unexpected content, and do not disable TLS verification. Keep credentials in private app data with restrictive permissions, never a manifest, asset or log.

A background server must declare its incoming port, bind deliberately (prefer loopback), authenticate any LAN control surface, retain connections, and await listener shutdown. Port declarations are collision checks, not a firewall opening. The network-service starter is loopback-only, caps responses, skips overlapping polls and does not retain response bodies.

Use context.timers while active, not global repeating timers that the host cannot track. Use asynchronous I/O for nontrivial work; large synchronous reads block Jibo's UI and speech coordination. Every allocated socket, stream, file descriptor, timer and child process belongs in the shutdown inventory.