roslaunch shutdown error when generating datasets from existing configs

[LarryDong]

Hi TimoStoff,

I am using your existing configs to generate bags. But when I ran python scripts/generate_preset.py /tmp, I got the following output (see the end):

I only get the first bag, but the following bags cannot be generated. But when I stop the program by ctrl+C, the code can generate the next new bag but failed the generate the following.

I guess there's something wrong with "launch.shutdown()" in generate_preset.py, which did not stop the ros node cleanly. But I am not familar with stoping ros launch by this way.

Any ideas to solve this problem? Many thanks.

(the full output)

(==> /tmp/000000000_config2d.txt')
...
publish image from callback
publish image from callback
publish image from callback
I0714 20:16:56.509268 8885 rosbag_writer.cpp:62] Finalizing the bag...
I0714 20:16:56.634008 8885 rosbag_writer.cpp:64] Finished writing to bag: /tmp/000000000_out.bag
[esim_node-1] process has finished cleanly
log file: /home/larrydong/.ros/log/e5db78ac-e497-11eb-a8f1-54b2039462bb/esim_node-1.log
[WARN] [1626265016.706444]: esim_node-1 died with code 0
[optic_flow_viz-3] killing on exit
[dvs_renderer-2] killing on exit*
(==> /tmp/000000001_config2d.txt')
Saving launch file to /tmp/esim.launch
... logging to /home/larrydong/.ros/log/e5db78ac-e497-11eb-a8f1-54b2039462bb/roslaunch-NUC-8863.log
started roslaunch server http://NUC:36919/
SUMMARY
PARAMETERS
/optic_flow_viz/arrows_scale: 0.07
/optic_flow_viz/arrows_step: 7
/optic_flow_viz/arrows_upsample_factor: 1
/optic_flow_viz/publish_rate: 100
/rosdistro: melodic
/rosversion: 1.14.10
NODES
/
dvs_renderer (dvs_renderer/dvs_renderer)
esim_node (esim_ros/esim_node)
optic_flow_viz (esim_visualization/optic_flow_converter.py)
ROS_MASTER_URI=http://localhost:11311
process[esim_node-4]: started with pid [8932]
process[dvs_renderer-5]: started with pid [8933]
process[optic_flow_viz-6]: started with pid [8934]
I0714 20:16:57.325270 8932 imp_multi_objects_2d_renderer.cpp:44] width = 256, height = 256
I0714 20:16:57.325460 8932 imp_multi_objects_2d_renderer.cpp:45] tmax = 10
I0714 20:16:57.325490 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/coco/000000425702.jpg.
I0714 20:16:57.328163 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.329393 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5a019a827ca233f48ba62757.png.
[optic_flow_viz-6] killing on exit
[dvs_renderer-5] killing on exit
I0714 20:16:57.330996 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
[esim_node-4] killing on exit
W0714 20:16:57.331701 8932 signal_handler.cpp:46] Signal handler was called with signal 2
I0714 20:16:57.332190 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Jersey-PNG-Transparent.png.
Traceback (most recent call last):
File "/home/larrydong/sim_ws/src/rpg_esim/event_camera_simulator/esim_visualization/src/py/optic_flow_converter.py", line 9, in
import rospy
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/init.py", line 47, in
from std_msgs.msg import Header
File "/opt/ros/melodic/lib/python2.7/dist-packages/std_msgs/msg/init.py", line 1, in
from ._Bool import *
File "/opt/ros/melodic/lib/python2.7/dist-packages/std_msgs/msg/_Bool.py", line 6, in
import genpy
File "/opt/ros/melodic/lib/python2.7/dist-packages/genpy/init.py", line 34, in
from . message import Message, SerializationError, DeserializationError, MessageException, struct_I
File "/opt/ros/melodic/lib/python2.7/dist-packages/genpy/message.py", line 48, in
import yaml
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/site-packages/yaml/init.py", line 8, in
from loader import *
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/site-packages/yaml/loader.py", line 4, in
from reader import *
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/site-packages/yaml/reader.py", line 47, in
class Reader(object):
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/site-packages/yaml/reader.py", line 147, in Reader
NON_PRINTABLE = re.compile(NON_PRINTABLE)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/re.py", line 194, in compile
return _compile(pattern, flags)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/re.py", line 249, in _compile
p = sre_compile.compile(pattern, flags)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 576, in compile
code = _code(p, flags)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 561, in _code
_compile(code, p.data, flags)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 107, in _compile
_compile_charset(av, flags, code, fixup, fixes)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 232, in _compile_charset
flags & SRE_FLAG_UNICODE):
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 274, in _optimize_charset
r = range(av[0], av[1]+1)
KeyboardInterrupt
I0714 20:16:57.339816 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.341230 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Gabriella-Wilde-Transparent-PNG.png.
I0714 20:16:57.349253 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.351073 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5c6404aee4b8dd029ff25b55.png.
I0714 20:16:57.352191 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.352732 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Human-PNG-Free-Download.png.
I0714 20:16:57.367199 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.369637 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Secretary-PNG-Transparent.png.
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: cHRM chunk does not match sRGB
I0714 20:16:57.387706 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.391546 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Abraham-Lincoln-Transparent-Background.png.
I0714 20:16:57.401508 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.402863 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5aead1916554160a79be9faa.png.
I0714 20:16:57.403936 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.404393 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/580b585b2edbce24c47b27b2.png.
I0714 20:16:57.405547 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.406095 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/580b585b2edbce24c47b2923.png.
I0714 20:16:57.407203 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.407754 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Hand-PNG-Transparent-File.png.
I0714 20:16:57.422911 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.425623 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5954bf12deaf2c03413be360.png.
I0714 20:16:57.426824 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.427173 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Walrus-PNG-Clipart.png.
I0714 20:16:57.435050 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.435981 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Lawyer-Transparent-PNG.png.
I0714 20:16:57.438027 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.438702 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Tie-Transparent-Background.png.
I0714 20:16:57.442828 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.443619 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Kingfisher-Background-PNG.png.
I0714 20:16:57.452098 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.454032 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5996e4a99c43bb35d25d73a5.png.
I0714 20:16:57.455160 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.482300 8932 ros_publisher.cpp:37] Initializing ROS
I0714 20:16:57.513383 8932 rosbag_writer.cpp:39] Will write to bag: /tmp/000000001_out.bag
W0714 20:16:57.513398 8932 adaptive_sampling_benchmark_publisher.cpp:57] Empty benchmark folder string: will not write benchmark files
W0714 20:16:57.513438 8932 synthetic_optic_flow_publisher.cpp:41] Empty output folder string: will not write synthetic optic flow files
I0714 20:16:57.715121 8932 rosbag_writer.cpp:62] Finalizing the bag...
I0714 20:16:57.715297 8932 rosbag_writer.cpp:64] Finished writing to bag: /tmp/000000001_out.bag

[frothmoon]

Hello , I also encountered a similar problem, have you solved this problem?

[LarryDong]

Hello , I also encountered a similar problem, have you solved this problem?

No I didn't try to fix this problem, but "avoided" this issue.

I wrote a script. Each time, this code only processed one bag data and stopped naturally. Then the script started the code to process the next bag data. Good luck.

[frothmoon]

Hello , I also encountered a similar problem, have you solved this problem?

No I didn't try to fix this problem, but "avoided" this issue.

I wrote a script. Each time, this code only processed one bag data and stopped naturally. Then the script started the code to process the next bag data. Good luck.

Thank you, I am also generating bag files one by one.

[gaopinghai]

I think it just the flag variable process_generate_running was not set true in function init_launch.

[maqscui]

@frothmoon @gaopinghai can you help me in clarifying a confusion? I have a confusion regarding the JSON files provided in generator_config folder. There are three paths you set.

"foreground_images": {
"png_dataset": {
"path": "/home/path/to/esim_objects",
"proportion": 0.8
},
"coco_dataset": {
"path": "/home/path/to/coco_png",
"proportion": 0.2
},
And then
"background_images": {
"coco_dataset": {
"path": "/home/path/to/coco"
}
Here I replaced "path": "/home/path/to/coco_png", this with png images that i extracted using tools/jpg_png.py script and "path": "/home/path/to/coco" with the original COCO training dataset that has jpg images.
What is this path "path": "/home/path/to/esim_objects", about? Which folder I should specify in this path?

[LarryDong]

@frothmoon @gaopinghai can you help me in clarifying a confusion? I have a confusion regarding the JSON files provided in generator_config folder. There are three paths you set.

"foreground_images": { "png_dataset": { "path": "/home/path/to/esim_objects", "proportion": 0.8 }, "coco_dataset": { "path": "/home/path/to/coco_png", "proportion": 0.2 }, And then "background_images": { "coco_dataset": { "path": "/home/path/to/coco" } Here I replaced "path": "/home/path/to/coco_png", this with png images that i extracted using tools/jpg_png.py script and "path": "/home/path/to/coco" with the original COCO training dataset that has jpg images. What is this path "path": "/home/path/to/esim_objects", about? Which folder I should specify in this path?

I think you can use any images here.

The author use two datasets (coco_png and esim_objects) for foreground_images.
Two images occupy 20% and 80% of the overall data.