Panther: OS for Z2 Mini

Software
1

Hello,

We are revisiting again ROS2 for our Panther. We have a Z2 mini as our user computer. Is this now compatible with the Husarion OS? Last time we tried, we couldn’t get the install to work on the Z2.

Thank you.

2

Hello @Tim_RMIT,

Generally, I see no reason why it would be necessary to install Husarion OS on a user’s computer.

In order for ROS communication to take place, it is necessary to ensure:

In terms of hardware:

  • ethernet connections

In terms of software:

  • appropriate version of Ubuntu (for version humble-2.1.1 it is Ubuntu 22.04).
  • ROS installation
  • time synchronization
  • DDS configuration. We both use cyclone by default, sometimes you need to specify a network interface to obtain data.
  • It is worth setting a static IP address to 10.15.20.3/24

Once the above aspects are met, any computer should be able to communicate with Panther.

Regards

3

Hello @RafalGorecki ,

I think my question may have been mis-understood. I refer to the ‘user computer’ inside Panther (as opposed to the ‘built-in’ Pi). Our model is the PC01 HP Z2 Mini.

For clarity, When we last looked into upgrading Panther to ROS 2 in May this year, the Husarion OS upgrade on Panther was a blocker. The current OS reinstall current lists only the NUC and Lenovo. While both refer back to the same OS image, I would like to confirm that the Z2 will also work (in non-headless mode) before attempting a re-install and preventing lost time if we have to revert again.

I appreciate it may also be possible to just upgrade the Z2 manually to Ubuntu 22.04 and ROS2, however, the Husarion OS comes with preconfigured docker containers and configuration for Panther which are quite beneficial, and on which our existing software stack depends.

Thank you and Kind Regards,
Tim

4

Hello @Tim_RMIT,

Thank you for the link to the previous thread and I understand your hesitations and doubts. I will briefly explain the broader perspective so that you understand the company’s decisions.

Husarion OS was created for ROSbots, where its presence is justified, because it configures clearly defined hardware located in the robot. In the case of Panther with ROS 2, the hardware is configured on the Built-In Computer, so the User Computer can be any hardware purchased by the user. Maintaining the system for subsequent devices would be time-consuming, and the licenses of some manufacturers make the process of introducing changes difficult. It makes more sense to base it on a known operating system and write instructions on what to provide to be able to communicate with the Built-in computer.

The difference between pure Ubuntu22.04 and Husarion OS comes down to the installation of a dozen or so packages, which the user can easily install themselves. There is nothing to prevent you from installing docker on a new system and using the same compos. Thanks to this approach, we do not impose a single path on the user to create their own application. The user can choose the most convenient form for them.

Perhaps Husarion OS will work with your computer, but it is withdrawn from sale and we are no longer conducting any tests on it.

Please be understanding. If you have any problems, please contact us.
Regards

5

Hello @RafalGorecki

Hello …,

Thank you for the additional information. I understand that Husarion are moving away from the custom OS for the built-in computer, and now the recommended approach is to manually install/upgrade and configure the OS.

However, I have some follow-up queries. Is there documentation (apologies if I’ve perhaps missed if) that details:

  • Recommended list of husarion (or other) docker containers (or repositories) for running default software with our Panther hardware (Velodyne Puck, Zed2, UR5e). I can find a reference of panther-navigation, but this depends on other packages (velodyne). Is there a complete list?
  • OS configuration that is required, including environment settings, and ubuntu software packages to leverage?

This would be useful as the most convenient approach for us is to leverage an existing functional software stack (and modify as needed), to then focus on our own research software.

Regards,
Tim

6

Hello @Tim_RMIT,

Thank you for your inquiry. In fact, there is one place missing in our documentation that would provide information on the question asked. However, I will describe the recommended approach to your sensors.

Sensor Assumptions

Sensor Demos
For most simple sensors, we have Docker images that will allow for quick testing and acquisition of sensor data. However, Docker images are not the best for devices that the user may want to configure custom to their preferences. In such situations, I recommend installing the package according to the manufacturer’s recommendations. Basic sensor integrations can be found at ROS Equipment | Husarion.

How to integrate devices
Our development team is striving for software in which the RPI is an integral part of the robot and the user does not need to access this computer. Therefore, changes that are not related with the Panther should not be made on Built-In Computer. Therefore, I recommend that all external devices are integrated and run from the user computer.

Your sensors

Velodyne
You can find velodyne Docker in this repository. Currently, there is a minor bug between the ros2 branch and ros2-tf-fix. Therefore, I recommend using the ros2-tf-fix branch.

ZED2
I upload the article and the corresponding repository to the docker containing ZED.
ZED images are strongly linked to NVIDIA drivers, which may require a driver update, and the ZED image itself is quite large. The manufacturer also provides separate tools for the camera and extensive configuration options. Personally, I would prefer a native installation of ZED.

UR5e
We had a demo for UR manipulators prepared for ROS1. As of today, we do not have a ready demo for the manipulator. Here are some useful links:

OS Configuration

I think all the most important information was presented in the first message.

Let me know if everything is clear or if you encounter any problems in our demos.

Regards

7

Hello @RafalGorecki,

Thank you for the detailed information.

It looks like it will take a bit of time for us to work through the different components. If we have follow-up questions, we’ll get back here.

Thank you for the information and assistance.
Tim

8

Hello @RafalGorecki,

I’m following up on the install, as I’ve been slowly working through various setup items.
We successfully installed the OS for the built-in computer, and updated our internal user computer. We by-and-large have the sensors (velodyne & zed) working, but I haven’t got to the UR.

However, we have run into a fairly major problem. (I hadn’t initially noticed this on re-installing the built-in computer). It seems to be the same issue from this recent thread, but expanded beyond just the IMU.

The issue appears to be that the hardware controller manager isn’t reporting data or publishing the expected topics. We are not getting topics/data/transform-frames for:

  • Wheels
    • The TF frame publishing for the wheels doesn’t report. More specifically, One frame gets published, and then nothing further, so the transform frame expires. (see screenshot).
    • There is no output on the topic /panther/odometry/wheels, and only the ekf_filter node as a subscriber, so it’s only created by the EFK filter node.
    • This results in an error No transform from [/panther/fl_wheel_link] to [panther/odom]
    • It seems, after re-launching the default panther_ros node multiple times that the hardware controller launches, publishing something, and then doesn’t publish further.
  • IMU
    • We don’t get any IMU publishers.
    • The /panther/imu/data topic sometimes appears, but only with ekf_filter node as a subscriber
    • The result is mapping packages (such as cartographer) that require the IMU report that no IMU data can be found.

However, what is strange is that we do get filtered odmoetry on the topic /panther/odometry/filtered. We also get a valid odom TF frame.

Any assistance would be greatly appreciated.

Some additional de-bugging info.

  • Our Panther version is v1.2
  • I am using the docker image: husarion/panther:humble-2.1.2-20241125, without modification (for now), and the default bring_up script.
  • I have spotted two ‘GPIO controller’ errors in the logs which may/may-not be related
panther_ros               | [ros2_control_node-1] NMT: entering operational state
panther_ros               | [ros2_control_node-1] error: SDO abort code 05040000 received on upload request of object 1000 (Device type) to node 01: SDO protocol timed out
panther_ros               | [ros2_control_node-1] error: SDO abort code 05040000 received on upload request of object 1000 (Device type) to node 02: SDO protocol timed out
  • I have run sudo pth_hw_update_config multiple times, and the machine reports that the hardware is up-to-date. However, very occasionally when I reboot the machine from power-on, I get the incorrect hardware warning.
  • Available topics (on .2) are:
/panther/cmd_vel
/panther/diagnostics
/panther/dynamic_joint_states
/panther/hardware/e_stop
/panther/hardware/io_state
/panther/hardware/motor_controllers_state
/panther/joint_states
/panther/joy
/panther/joy/set_feedback
/panther/robot_description
/panther/system_status
/panther/velodyne/scan
/panther/velodyne/velodyne_packets
/panther/velodyne/velodyne_points
9

Hello @Tim_RMIT,

I suspect that the problem lies with ROS Middleware. To confirm this, firstly please log in to the built-in computer and check if all topics are available from the built-in computer. If everything looks fine on the built-in computer. Please make sure that cyclonedds is installed and set on the user computer.

sudo apt install ros-$ROS_DISTRO-rmw-cyclonedds-cpp
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp

If it turns out that despite the cyclone dds setting, not all topics from the built-in computer are visible. I would ask you to familiarize yourself with the cyclone configuration. Setting the <NetworkInterface/> may be particularly helpful.

I am waiting for information whether everything works correctly in the built-in computer.
Regards

10

Hello @RafalGorecki ,

Thank you for the quick reply.

The topics I listed, and the issues with data are from the built-in computer. I can also confirm that all machines have the DDS property set, and that the DDS variable is set for all of the docker containers.

Regardless, I can additionally look into the DDS configuration file. However, do you have a preferred list of settings? I am unsure what I should be looking to configure on the network interfaces. If a default configuration file used in the husarion docker images?

Thank you.

11

I’m not sure if I understood the statement correctly.

The topics I listed, and the issues with data are from the built-in computer.

  1. So are all topics available from documentation are available from the build-in computer? Does the data also come in?
  2. Can you make talker listener test in both way User Copmuter → Built-In Computer and Built-In Computer → User Copmuter?

Useful commands:

sudo apt-get install ros-$ROS_DISTRO-demo-nodes-cpp
ros2 run demo_nodes_cpp talker
ros2 run demo_nodes_cpp listener
12

Hello @RafalGorecki ,

Thank you again, for the ongoing help. This issue is doing my head in.

I had meant I was looking at the topics and outputs of the topics on the built-in computer to avoid all potential networking issues. I wasn’t seeing any data on the wheels/imu topics on the built-in machine.

However, I think this first issue might have been an issue with the ZED node and conflicts with it’s robot_state_publisher node. I don’t really see why that’s an issue, but after moving all the ZED URDF to the panther_description package, all topics correctly publish and receive on the built-in Pi (.2) and our on-board user computer (.3).

Unfortunately, we additionally have communication issues to my external computer over the Panther wifi network. (So I think this is part of the confusion.) I agree that your suspicion on the DDS communication. To note, I’ve had rmw_cyclonedds_cpp configured. However, something doesn’t make much sense to me:

  • On the built-in (pi .2) and on-board user computer (.3), the docker containers all have the same config with DDS of rmw_cyclonedds_cpp. The full env config is below
x-common-config: &common-config
  network_mode: host
  ipc: host
  restart: always
  environment:
    - RMW_IMPLEMENTATION=rmw_cyclonedds_cpp # Default FastDDS do not work
    - ROBOT_MODEL=${ROBOT_MODEL:-panther}
    - ROBOT_NAMESPACE=${ROBOT_MODEL:-panther}
    - ROS_DOMAIN_ID=${ROS_DOMAIN_ID:-0}
  • My external machine is a VM running Ubuntu 22.04, with bridged networking.
  • When export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp, I can always get the topic list (ros2 topic list), but sometimes topics send (via ros2 topic echo), and sometimes they don’t. Also when they send, it’s very slow with a lot of latency.
  • If I remote the DDS config, (unset RMW_IMPLEMENTATION), everything receives fine (and a lot faster).

So I’m quite at a loss of the issue. It’s almost like cyclone DDS is much worse.
I’ve no idea what I should be configuring on my local machine. Is there a standard DDS config file for use with Husarion bots?

So any assistance is greatly appreciated.

13

Hello @Tim_RMIT,

It’s hard for me to say what exactly might be causing the problem in your setup.

  1. Personally, I would start with a minimal talker listener time and check if there is access to topics from all available devices: user-computer, built-in computer, your laptop.

  2. You can also try to explicitly specify the interfaces over which the information is to be propagated. Example:

<?xml version="1.0" encoding="UTF-8" ?>
<CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
  <Domain Id="any">
    <General>
      <Interfaces>
        <NetworkInterface name="eth0"/>
        <NetworkInterface name="wlan0"/>
      </Interfaces>
    </General> 
  </Domain>
</CycloneDDS>
  1. You can also try using fastdds, preferably with discovery server. Thanks to discover server, the bandwidth load should be reduced. However, it requires additional configuration. More information in ROS2 Documentation.

Most likely in connection with the proclamation, we will create some blog post. Since there is a lack of information about DDS on the web and it would be possible to make a useful material for our community.

If you have any problems, I can help. If you find a solution, we would also appreciate the information.

Regards