Cameras

The camera class in RCareWorld is in pyrcareworld/sensors/camera.py. As you can see, the camera class is inherited from RCareWorldBaseObject, so it has all the functionalities and properties owned by BaseObjects.

Similar to BaseObjects, you can set the position and rotation, load, copy, and also destroy the camera. For details, please check the object section.

Initialize a camera

To initialize a camera, you need to provide one of the two kinds of information

Intrinsic matrix: It should be a 9-element list. For instance [600, 0, 0, 0, 600, 0, 240, 240, 1].
Width, Height, FOV (optional)

# Example:
intrinsic_matrix = [600, 0, 0, 0, 600, 0, 240, 240, 1]
camera = env.create_camera(id=1234, 'example_camera', intrinsic_matrix)

Initialize data and get data

In the camera class, the data acquisition is divided into two steps, namely initialize and get. The initialize step informs the Unity side that the user now wants to have access to this kind of data information, and then Unity will start to send this information to Python side.

For initialize, there are three ways to specify the camera parameters similar to Initialing a camera.

RGB data

To get RGB data, use initializeRGBWithIntrinsic or initializeRGB. Don't forget to do a getRGB after initializing.

# Example
# Initializa RGB, this function only needs to be called once
# Call this function if the camera is initialized by the intrinsic matrix
camera.initializeRGBWithIntrinsic()
# Or call this function if the camera is initialized by height, width, or height, width and fov
camera.initializeRGB()
# Get RGB data
rgb_img = camera.getRGB()

Depth data

Similarly, use initializeDepthWithIntrinsic or initializeDepth, and getDepth. This one returns perfect depth generated by Unity's shader.

# Example:
# Initializa Depth, this function only needs to be called once
# Call this function if the camera is initialized by the intrinsic matrix
camera.initializeDepthEXRWithIntrinsic()
# Or call this function if the camera is initialized by height, width, or height, width and fov
camera.initializeDepthEXR()

ActiveDepth data

Following SAPIEN2 and pyrfuniverse, we now have IR based active depth sensor.

# Example:
# Initializs initializeActiveDepthWithIntrinsic, this function only needs to be called once
# Call this function if the camera is initialized by the intrinsic matrix, and you need to pass the intrinsic matrix for ir sensor
camera.initializeActiveDepthWithIntrinsic(ir_intrinsic_matrix)
camera.getActiveDepth()