Light flares/sprites visible in Camera sensor despite no car is there

[alessiogambi]

We noticed that the camera sensor while the ego-car is breaking (or going backward) produces images where a red/white light is visible (biasing AI/DL predictions).

You can see the attached images for an example. Note I clipped the images to fit in this page.

These images have been collected using the following configuration:

I'm using Python 3.7, Windows 10 64 Bit, BeamNGpy 1.15, BeamNG.research unlimited revision #45 (#47 is the newest)

[alessiogambi]

@masskro0 might be interested (as he created those images in the first place ;) )

[pascale-commits]

Could you post the code that produced these images?

[alessiogambi]

Guess that @masskro0 can do that 👍

[masskro0]

Sorry for the late reply. Here is the code that I used:

# Setup camera.
direction = (0, 1, 0)
fov = 90
resolution = (1280, 720)
x, y, z = -0.3, 2.1, 1
camera = Camera((x, y, z), direction, fov, resolution, colour=True, depth=True, annotation=True)
ego.attach_sensor("camera", camera)
beamng = BeamNGpy('localhost', 64286)
bng = beamng.open()
bng.load_scenario(converter.scenario)
bng.start_scenario()
while True:
    sensors = bng.poll_sensors(ego)
    ego.update_vehicle()
    img = sensors["camera"]["colour"].convert("RGB")
    filename = label + '_{}.png'.format(time())
    file_path = join(image_dir, filename)
    img.save(file_path)

I'm going to check whether this problem occurs with the newest BeamNG.research version as well.

[alessiogambi]

@aivora-beamng I experienced the same issue also in version BeamNG.tech.v0.31.3.0. See image below.

Since we plan to use this vehicles for testing/training vision-based systems, the sprites creates a bias in the data

[Quant14]

@alessiogambi Where is the camera positioned relatively to the ego car? Also does this happen in DX11, Vulkan or both? We use the camera in a dashcam position for the Lane-keeping assist and haven't been able to reproduce these sprites.

[alessiogambi]

@Quant14 Could you check if this code shows the issue?
#73 (comment)

[Quant14]

@alessiogambi Where to place the sensor though, in your code it seems to be near the rear, looking backwards. I tried the ETK800 but the camera clips inside the vehicle partially. No sprites either, from my incorrect placement anyway. Also no sprites when sticking the camera to the back of the car. When the camera is in the front and looking forward, there's again no sprites. It's possible that in the more recent versions of BeamNG.tech this was fixed. Let me know if you need an updated version of your code that works for the newest version of the camera sensor.

[alessiogambi]

@Quant14 it could be that a newer release of BeamNG.tech fixed it. I'll try to run some more experiments with the latest BeamNG.tech release and report an update on this issue.