Robora's Synths

Introduction to Synths

In our infrastructure, Synths are fundamentally autonomous software agents designed to perform tasks through a continuous perceive-reason-act cycle. Robora Synths are specialized robotics executors, making them essentially "smart brains" you can embed into physical robots or IoT devices to make them autonomous and efficient. They're designed for real-world tasks like grasping objects, navigating spaces, or assembling parts using Vision-Language-Action (VLA) models to process sensor data, reason about instructions, and generate actions. Unlike general AI agents, Synths are robotics-only, focusing on physical control to unify fragmented data and software in the ecosystem. They run locally on robot hardware for low-latency performance but scale through the Grid Framework's coordination.

In short, what are Synths about?

As earlier stated, Synths are fundamentally autonomous software agents. Unlike traditional "AI Agents", Synths are imbedded with a VLA-model with optional "plus" tools for a continuous perceive-reason-act cycle:

• Perceive: Gathers input from robot sensors (e.g., cameras, LiDAR) to "see" the environment.

• Reason: Analyzes data with natural-language instructions (e.g., "grasp the red object"), using VLA to prune unnecessary info and schedule efficient actions.

• Act: Outputs commands to hardware (e.g., move an arm via ROS interfaces).

Synths are created in the Grid layer of the Grid Framework. In the 3D-Builder, you can drag-and-drop them into robot designs. They're lightweight, running on edge devices like NVIDIA Jetson boards, and can be customized with tools for specific needs (e.g., energy profiling for battery-limited bots).

Users can drag-and-drop Synths into custom robot assemblies, embedding them alongside hardware components like sensors and actuators. Once deployed, a Synth operates locally on the robot's hardware (e.g., via Docker-isolated runtimes on edge devices such as NVIDIA Jetson boards), ensuring low-latency performance in dynamic settings. For instance:

• Perception Phase: Synths ingest sensor data (e.g., camera feeds or LiDAR scans) and apply SP-VLA's pruning to filter redundant spatial information, focusing on salient features like object contours for tasks such as grasping.

• Reasoning Phase: Leveraging VLA's adaptive scheduling, Synths classify actions, enabling intelligent decision-making without excessive compute.

• Action Phase: Outputs are translated into direct commands for physical actuators (e.g., via ROS2 interfaces), supporting real-time control of IoT devices or robotic arms.

Synths are extensible, allowing "plus" system-level tools to be layered on for enhanced features, such as communication plugins for multi-robot swarms (e.g., pub/sub for coordinated assembly) or energy profilers for battery-constrained mobile robots. This modularity aligns with Robora's emphasis on sustainability.

Possibility to "train" your own Synth

Training a Synth involves fine-tuning its underlying Vision-Language-Action (VLA) model with robotics data to improve its perceive-reason-act cycle (perceive sensors, reason/prune data, act on hardware). It's integrated into the Grid Framework for efficiency, using gamified data collection and the marketplace. Here's a simple breakdown

• Gather Data (Intake Grid Layer): Start with real-world or simulated data. Use Robora's gamified tools (e.g., browser-based teleoperation games where you control a virtual robot to "teach" actions like grasping, earning $RBR points for quality contributions (see roadmap) ). Fill questionnaires for human-labeled insights (e.g., "Was this path clear? Rate the obstacle avoidance"). Or upload your own sensor logs/packages (e.g., camera feeds from tests). Data can be bought from the marketplace if you need more variety. This step unifies fragmented robotics data, making it ready for training.

• Customize and Fine-Tune (Fusion Grid Layer): In a training interface (linked to the 3D builder or standalone app), select a base Synth (pre-trained VLA wrapper) and import your data. Fine-tune the VLA model (e.g., adjust parameters for better grasping in low-light). Use "plus" tools if needed (e.g., add a simulation plugin for virtual testing). The off-chain Nexus Hub matches resources like GPUs (via DePIN for cheap sharing) to handle the compute, keeping it fast and low-cost.

• Test and Scale (Propulsion Grid Layer): Run the trained Synth in a simulation (Gazebo) (e.g., via ROS2 in our robora-ros-sims repo) or on a test robot to validate.

• Verify and Share (Emission Grid Layer): The framework verifies performance (e.g., off-chain proofs confirm improved accuracy). This loops feedback. As you've been training your Synth, you can upload the trained Synth to the marketplace for $RBR rewards (e.g., sell as a "grasp package" that others can buy and embed or further train). The Synth forms a shareable asset.

Best use cases of Synths: Fleet Control

Synths are ideal for fleet control (e.g., coordinating 10-100 robots in a warehouse), where each robot runs a Synth based on a Hybrid Architecture. Synths will run on robots with edge-local execution and on a semi-centralized Nexus Hub (hosted on private clouds) for independence. Local 5G/Wi-Fi meshes will be used for intra-fleet communications, while falling back to cloud only for non-real-time tasks like logging.