Set Up Unity ROS TCP Connector in 12 Minutes

Connect Unity to ROS2 with the TCP Connector package. Fix common setup errors and achieve reliable message passing for robotics simulation.
·
4 min read
·
Mark
·
Robotics

Problem: Unity Won't Connect to Your ROS2 Node

You're building a robot simulation in Unity and need to send sensor data to ROS2, but the TCP Connector keeps timing out with Connection refused or messages never arrive.

You'll learn:

  • How to configure TCP Connector for ROS2 Humble/Iron
  • Fix the "10s timeout" connection error
  • Verify bidirectional message flow

Time: 12 min | Level: Intermediate

Why This Happens

The ROS TCP Connector uses a client-server model where Unity acts as the TCP client. If the ROS TCP Endpoint server isn't running first, or if IP/port config mismatches, Unity fails silently after timeout.

Common symptoms:

  • Unity console shows [ROS] Waiting for ROS connection... indefinitely
  • Server starts but no messages appear in ros2 topic echo
  • Works locally but fails on network/Docker setups

Solution

Step 1: Install ROS TCP Endpoint Package

# In your ROS2 workspace
cd ~/ros2_ws/src
git clone https://github.com/Unity-Technologies/ROS-TCP-Endpoint.git
cd ~/ros2_ws
colcon build --packages-select ros_tcp_endpoint
source install/setup.bash

Expected: Build succeeds with no errors. If colcon fails, check ROS2 is sourced.

If it fails:

  • Error: "package 'ros_tcp_endpoint' not found": Run rosdep install --from-paths src --ignore-src -y first
  • Python errors: Endpoint requires Python 3.8+, verify with python3 --version

Step 2: Start the TCP Endpoint Server

# Default: localhost:10000
ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_IP:=0.0.0.0

# For specific port
ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_IP:=0.0.0.0 -p ROS_TCP_PORT:=10000

Why 0.0.0.0: Binds to all network interfaces. Use 127.0.0.1 only for same-machine testing.

You should see:

Starting ROS TCP Server...
Listening on 0.0.0.0:10000

Keep this Terminal open - server must run before Unity connects.

Step 3: Configure Unity TCP Connector

In Unity Editor:

  1. Install via Package Manager:

    • Window → Package Manager → Add package from git URL
    • Enter: https://github.com/Unity-Technologies/ROS-TCP-Connector.git?path=/com.unity.robotics.ros-tcp-connector

  2. Configure ROSConnectionPref:

    // Create in any GameObject
using Unity.Robotics.ROSTCPConnector;

void Start()
{
    // Before ANY ROS operations
    ROSConnection.GetOrCreateInstance().Connect(
        "192.168.1.100",  // ROS machine IP
        10000              // Port matching server
    );
}

  3. Or use Robotics → ROS Settings UI:

    • ROS IP Address: 192.168.1.100 (your ROS machine)
    • ROS Port: 10000
    • Protocol: ROS2
    • Tick "Connect on Startup"

Expected: Unity console shows [ROS] Connection established within 2 seconds.

Step 4: Test Message Publishing

using Unity.Robotics.ROSTCPConnector;
using RosMessageTypes.Std;

public class TestPublisher : MonoBehaviour
{
    private ROSConnection ros;
    private float publishTimer = 0f;
    
    void Start()
    {
        ros = ROSConnection.GetOrCreateInstance();
        // Register publisher - topic auto-created in ROS
        ros.RegisterPublisher<StringMsg>("unity_test");
    }
    
    void Update()
    {
        publishTimer += Time.deltaTime;
        
        if (publishTimer > 1f)  // Publish every 1s
        {
            StringMsg msg = new StringMsg
            {
                data = $"Unity time: {Time.time:F2}"
            };
            
            ros.Publish("unity_test", msg);
            publishTimer = 0f;
        }
    }
}

Why this works: RegisterPublisher creates the topic in ROS2. Messages serialize to ROS format automatically.

If it fails:

  • No error but messages don't appear: Check server terminal for connection logs
  • "Topic not registered": Call RegisterPublisher before Publish
  • "Type mismatch": Verify RosMessageTypes.Std package is imported

Step 5: Subscribe to ROS Topics

using RosMessageTypes.Geometry;

void Start()
{
    ros = ROSConnection.GetOrCreateInstance();
    
    // Subscribe to existing ROS topic
    ros.Subscribe<TwistMsg>("cmd_vel", ReceiveVelocity);
}

void ReceiveVelocity(TwistMsg msg)
{
    // This runs on ROS message thread - be careful with Unity objects
    float linearX = (float)msg.linear.x;
    Debug.Log($"Received velocity: {linearX}");
    
    // For Unity operations, queue to main thread:
    UnityMainThreadDispatcher.Instance().Enqueue(() =>
    {
        transform.Translate(Vector3.forward * linearX * Time.deltaTime);
    });
}

Critical: ROS callbacks run on background threads. Access Unity API only via main thread dispatcher or you'll get crashes.

Verification

Test publishing:

# In ROS terminal
ros2 topic echo /unity_test

You should see:

data: 'Unity time: 1.23'
---
data: 'Unity time: 2.24'

Test subscribing:

# Publish to topic Unity subscribes to
ros2 topic pub /cmd_vel geometry_msgs/msg/Twist "{linear: {x: 0.5}}" -1

Unity console should log the velocity value immediately.

What You Learned

  • TCP Endpoint server must start before Unity connects
  • 0.0.0.0 binding allows network connections beyond localhost
  • ROS callbacks run on background threads - use dispatcher for Unity API
  • Topics auto-register on first publish in ROS2 mode

Limitations:

  • 100 Hz message rate limit (TCP overhead). Use UDP for higher rates.
  • Large messages (>1MB) can timeout - consider compression.
  • Doesn't support ROS2 QoS policies yet (reliability/durability).

Common Network Troubleshooting

Docker Setup

If ROS runs in Docker:

# Start endpoint with host network
docker run --network host ros:humble ros2 run ros_tcp_endpoint default_server_endpoint

In Unity, use host.docker.internal (macOS/Windows) or bridge IP (Linux).

Firewall Issues

# Ubuntu - allow port 10000
sudo ufw allow 10000/tcp

# Check if port is listening
ss -tuln | grep 10000

Multiple Unity Instances

Each needs a unique port:

// Unity Instance 1
ROSConnection.GetOrCreateInstance().Connect("192.168.1.100", 10000);

// Unity Instance 2
ROSConnection.GetOrCreateInstance().Connect("192.168.1.100", 10001);

Start separate endpoints:

ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_TCP_PORT:=10001

Performance Tips

Reduce Latency:

// Set update frequency (default: 60 Hz)
ROSConnection.GetOrCreateInstance().SetMessageBatchingInterval(16); // ~60 Hz

Message Pooling:

// Reuse message objects instead of creating new ones
private TwistMsg reusableMsg = new TwistMsg();

void PublishVelocity(float x)
{
    reusableMsg.linear.x = x;
    ros.Publish("cmd_vel", reusableMsg);
    // Don't create new TwistMsg() every frame
}

Conditional Publishing:

// Only publish when values change significantly
private float lastPublishedValue = 0f;

void Update()
{
    float currentValue = GetSensorReading();
    
    if (Mathf.Abs(currentValue - lastPublishedValue) > 0.01f)
    {
        PublishSensorData(currentValue);
        lastPublishedValue = currentValue;
    }
}

Tested on Unity 2023.2 LTS, ROS2 Humble, Ubuntu 22.04 & Windows 11