ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

OpenLMD, Multimodal Monitoring and Control of LMD processing

?Download as PPTX, PDF?
?
Jorge Rodr¨ªguez Ara¨²jo
Jorge Rodr¨ªguez Ara¨²jo

This paper presents OpenLMD, a novel open-source solution for on-line multimodal monitoring of Laser Metal Deposition (LMD). The solution is also applicable to a wider range of laser-based applications that require on-line control (e.g. laser welding). OpenLMD is a middleware that enables the orchestration and virtualization of a LMD robot cell, using several open-source frameworks (e.g. ROS, OpenCV, PCL). The solution also allows reconfiguration by easy integration of multiple sensors and processing equipment. As a result, OpenLMD delivers significant advantages over existing monitoring and control approaches, such as improved scalability, and multimodal monitoring and data sharing capabilities.

1 of 14
Downloaded 19 times
OpenLMD, Multimodal Monitoring and Control of LMD processing Jorge Rodr¨ªguez-Ara¨²jo AIMEN Technology Center, Porri?o, Spain Photonics WEST 2017, 1-2-2017
openlmd.github.io | jorge.rodriguez@aimen.es 2 Index Index 1. Laser Metal Deposition 2. Motivation and innovative character 3. Open Laser Metal Deposition 4. ROS-based robot cell integration 5. Multimodal monitoring and virtualization 6. Image registration and data acquisition 7. Closed-loop laser power control 8. 3D geometrical monitoring 9. Adaptive LMD path planning 10.Conclusions and future work
openlmd.github.io | jorge.rodriguez@aimen.es 3 Laser Metal Deposition (LMD) ? Promising additive manufacturing technique ? Parts are built up layer by layer directly from a 3D CAD model ? The material is directly deposited on the previous surface ? For repair and direct fabrication of pieces ? Near-net-shape (close to the final shape) ? Manufacturing of large metallic parts LMD issues ? Complex setup and adjustment of parameters ? Time consuming robot programming for repairing tasks ? Thermal heating accumulation and dimensional distortion ? Repeatability, rising defects, and metalurllical properties Traditional off-line process (with constant parameters) becomes unsucessful for large metallic parts LMD, Laser Metal Deposition
openlmd.github.io | jorge.rodriguez@aimen.es 4 Motivation and innovative character Motion Controller Off-line Path Programming 6-Axis Robot Laser Powder Feeder Power Controller Flow Controller Motivation ? Lots of industrial robotized laser cells ? Mostly manually operated ? Lack of integration for monitoring and control Innovation ? Retrofit current laser cladding facilities for LMD ? Empower robotized laser cells for effective AM ? Apply state of the art robotic software solutions Goals ? Build a modular architecture for LMD full automation ? Reduce heating accumulation for large parts ? Adapt robot programming to the real part ? Increase geometry accuracy and repeatability Conventional Robotized Laser Cladding Cell
openlmd.github.io | jorge.rodriguez@aimen.es 5 Open Laser Metal Depositon Concept and approach ? Open-source solution for on-line multimodal monitoring and control of LMD ? Modular set of software components. Built on ROS (Robot Operating System) ? Robotics, machine vision, embedded control, machine learning, big data ? Full compatible with current robotized laser cladding cells ? Focus on interoperability and standardization ROS-based architecture ? Multiprocessing architecture based on message publishing ? Multi-node and multi-machine ? Modular (e.g. robot, laser, camera) ? Synchronized data acquisition (common timestamp) ? High bandwidth data management (i.e. images) ? Visualization tools and components (e.g. rviz) ? Advanced robotics environment
openlmd.github.io | jorge.rodriguez@aimen.es 6 ROS-based robot cell integration Powder Feeder Fiber Laser 6-Axis Industrial Robot PC Controller Cladding Head ROS-Driver (ABB Rapid) Geometrical Cell Description (URDF) STATE PUBLISHER Laser Source (slave) Powder Feeder (slave) COMMAND SERVER Power Speed Powder flow Motion path States Commands ROBOT Process parameters AIMEN¡¯s LMD robotized laser cell ROS-based integration of modular laser cells ? The PC commands the robot integrating interfaces and modules with ROS ? The robot controls all the cell elements ROS components for robot integration ? Geometrical description (URDF) ? ROS driver
openlmd.github.io | jorge.rodriguez@aimen.es 7 Multimodal monitoring and virtualization Multimodal Cladding Head 3D System Tachyon MWIR NIR MWIR+NIR Multispectral Imaging LMD Cell Virtualization Multimodal monitoring approach ? Coaxial SWIR/MWIR images (thermal monitoring): NIT microcore (1000fps) [1-3um] ? Coaxial NIR images (surface monitoring): CMOS camera (100fps) [830-880nm] ? Off-axis 3D system: on-line 3D point cloud scanning (50fps)
openlmd.github.io | jorge.rodriguez@aimen.es 8 Coaxial sensors registration ? Image registration: process of transforming different sets of data into one coordinate system Data acquisition ? High throughput (28MB/s) ? NIR + MWIR + 3D point cloud + robot ? Data management and analysis ? Bag files and Pandas DataFrames NIT NIR (0, 0) vel x y Image registration and data acquisition Calibration ¡ú Projection
openlmd.github.io | jorge.rodriguez@aimen.es 9 Closed-loop laser power control ? High speed SWIR/MWIR thermal meltpool monitoring (1000fps) ? NIT Europe Tachyon 1024 microcore camera ? Meltpool geometrical monitoring (elliptical approximation) ? Increased geometry repeatability and reduced dilution and heat accumulation Closed-loop laser power control Meltpool, ellipse approach Width (mm) Time (s)
openlmd.github.io | jorge.rodriguez@aimen.es 10 3D triangulation working table nozzle 3D geometrical monitoring Industrial Robotic Laser Cell ROBOT ROS-DRIVER CAMERA IDS-DRIVER State Publisher Peak Finder Robot Pose Tool-Camera Laser TriangulationCalibration 3D ProfileCamera Pose 3D Point Cloud Working Cell Coordinate On-line 3D geometrical reconstruction On-line 3D point cloud registration ? Real-time point cloud registration ? Actual metric measurement (mm) ? Direct acquisition in robot coordinates
openlmd.github.io | jorge.rodriguez@aimen.es 11 Adaptive LMD path planning Adapts the path to the real geometry ? 3D vision guided full automated laser cladding repair of complex metallic parts ? Automatic generation of robot trajectories 1. Part scanning and filtering 2. Surface selection and path generation 3. Repair job generation and supervision 3D Filtering Initialization (setup) Scan layer Depth mapTarget Depth map Disparity Data Layer path planning Layer Path Planning (geometrical control) Laser Cell supervisor Robotized Cell 0 Finished Repair Job 3D geometrical control Coated surface
openlmd.github.io | jorge.rodriguez@aimen.es 12 Automatic coating of surfaces 3D scanning and robot path planning Enabled by the 3D point cloud directly provided by the 3D geometrical monitoring solution 1. Workarea scanning (direct part information in 1. cell coordinates) [mm] 2. 3D point cloud projection (2D Zmap image) 3. Surface selection directly in the 2D image 4. Segmentation of the Zmap image 5. Contours calculation from the segmented surface 6. Contours and Zmap feed the path planner 7. A new path is automatically calculated from that 1. information A second scanning after the process enables an adaptive path planning strategy in a full automatic way Surface selection 3D scanned surfaces
openlmd.github.io | jorge.rodriguez@aimen.es 13 Conclusions ? Integration and data acquisition ? Spatial reference system and temporally synchronized ? Monitoring and control ? Real-time closed-loop laser power control ? Adaptive path planning Work in-progress ? Embedded image monitoring and control ? Big data and deep learning approaches Robot ? Pose ? Process speed 3D geometry ? Point cloud (<0.5mm) SWIR/MWIR-NIR ? 2D melt pool geometry ? Thermal distribution and texture Conclusions and future work Reconfigurable Modular and reconfigurable Interoperability Large parts Low-cost solution Scalability
AIMEN ¨C Central y Laboratorios c/ Relva 27 A 36410 ¨C O PORRI?O (Pontevedra) Telf.+34 986 344 000 ¨C Fax. +34 986 337 302 Thank you for your attention Jorge Rodr¨ªguez-Ara¨²jo | Research Engineer Ph +34 986 344 000 | jorge.rodriguez@aimen.es www.aimen.es | aimen@aimen.es This work has receive funding from the European Union¡¯s Horizon 2020 research and innovation programme under grant agreement N? 637081. The dissemination of results herein reflects only the author¡¯s view and the Commission is not responsible for any use that may be made of the information it contains. http://openlmd.github.io

More Related Content

OpenLMD, Multimodal Monitoring and Control of LMD processing

  • 1. OpenLMD, Multimodal Monitoring and Control of LMD processing Jorge Rodr¨ªguez-Ara¨²jo AIMEN Technology Center, Porri?o, Spain Photonics WEST 2017, 1-2-2017
  • 2. openlmd.github.io | jorge.rodriguez@aimen.es 2 Index Index 1. Laser Metal Deposition 2. Motivation and innovative character 3. Open Laser Metal Deposition 4. ROS-based robot cell integration 5. Multimodal monitoring and virtualization 6. Image registration and data acquisition 7. Closed-loop laser power control 8. 3D geometrical monitoring 9. Adaptive LMD path planning 10.Conclusions and future work
  • 3. openlmd.github.io | jorge.rodriguez@aimen.es 3 Laser Metal Deposition (LMD) ? Promising additive manufacturing technique ? Parts are built up layer by layer directly from a 3D CAD model ? The material is directly deposited on the previous surface ? For repair and direct fabrication of pieces ? Near-net-shape (close to the final shape) ? Manufacturing of large metallic parts LMD issues ? Complex setup and adjustment of parameters ? Time consuming robot programming for repairing tasks ? Thermal heating accumulation and dimensional distortion ? Repeatability, rising defects, and metalurllical properties Traditional off-line process (with constant parameters) becomes unsucessful for large metallic parts LMD, Laser Metal Deposition
  • 4. openlmd.github.io | jorge.rodriguez@aimen.es 4 Motivation and innovative character Motion Controller Off-line Path Programming 6-Axis Robot Laser Powder Feeder Power Controller Flow Controller Motivation ? Lots of industrial robotized laser cells ? Mostly manually operated ? Lack of integration for monitoring and control Innovation ? Retrofit current laser cladding facilities for LMD ? Empower robotized laser cells for effective AM ? Apply state of the art robotic software solutions Goals ? Build a modular architecture for LMD full automation ? Reduce heating accumulation for large parts ? Adapt robot programming to the real part ? Increase geometry accuracy and repeatability Conventional Robotized Laser Cladding Cell
  • 5. openlmd.github.io | jorge.rodriguez@aimen.es 5 Open Laser Metal Depositon Concept and approach ? Open-source solution for on-line multimodal monitoring and control of LMD ? Modular set of software components. Built on ROS (Robot Operating System) ? Robotics, machine vision, embedded control, machine learning, big data ? Full compatible with current robotized laser cladding cells ? Focus on interoperability and standardization ROS-based architecture ? Multiprocessing architecture based on message publishing ? Multi-node and multi-machine ? Modular (e.g. robot, laser, camera) ? Synchronized data acquisition (common timestamp) ? High bandwidth data management (i.e. images) ? Visualization tools and components (e.g. rviz) ? Advanced robotics environment
  • 6. openlmd.github.io | jorge.rodriguez@aimen.es 6 ROS-based robot cell integration Powder Feeder Fiber Laser 6-Axis Industrial Robot PC Controller Cladding Head ROS-Driver (ABB Rapid) Geometrical Cell Description (URDF) STATE PUBLISHER Laser Source (slave) Powder Feeder (slave) COMMAND SERVER Power Speed Powder flow Motion path States Commands ROBOT Process parameters AIMEN¡¯s LMD robotized laser cell ROS-based integration of modular laser cells ? The PC commands the robot integrating interfaces and modules with ROS ? The robot controls all the cell elements ROS components for robot integration ? Geometrical description (URDF) ? ROS driver
  • 7. openlmd.github.io | jorge.rodriguez@aimen.es 7 Multimodal monitoring and virtualization Multimodal Cladding Head 3D System Tachyon MWIR NIR MWIR+NIR Multispectral Imaging LMD Cell Virtualization Multimodal monitoring approach ? Coaxial SWIR/MWIR images (thermal monitoring): NIT microcore (1000fps) [1-3um] ? Coaxial NIR images (surface monitoring): CMOS camera (100fps) [830-880nm] ? Off-axis 3D system: on-line 3D point cloud scanning (50fps)
  • 8. openlmd.github.io | jorge.rodriguez@aimen.es 8 Coaxial sensors registration ? Image registration: process of transforming different sets of data into one coordinate system Data acquisition ? High throughput (28MB/s) ? NIR + MWIR + 3D point cloud + robot ? Data management and analysis ? Bag files and Pandas DataFrames NIT NIR (0, 0) vel x y Image registration and data acquisition Calibration ¡ú Projection
  • 9. openlmd.github.io | jorge.rodriguez@aimen.es 9 Closed-loop laser power control ? High speed SWIR/MWIR thermal meltpool monitoring (1000fps) ? NIT Europe Tachyon 1024 microcore camera ? Meltpool geometrical monitoring (elliptical approximation) ? Increased geometry repeatability and reduced dilution and heat accumulation Closed-loop laser power control Meltpool, ellipse approach Width (mm) Time (s)
  • 10. openlmd.github.io | jorge.rodriguez@aimen.es 10 3D triangulation working table nozzle 3D geometrical monitoring Industrial Robotic Laser Cell ROBOT ROS-DRIVER CAMERA IDS-DRIVER State Publisher Peak Finder Robot Pose Tool-Camera Laser TriangulationCalibration 3D ProfileCamera Pose 3D Point Cloud Working Cell Coordinate On-line 3D geometrical reconstruction On-line 3D point cloud registration ? Real-time point cloud registration ? Actual metric measurement (mm) ? Direct acquisition in robot coordinates
  • 11. openlmd.github.io | jorge.rodriguez@aimen.es 11 Adaptive LMD path planning Adapts the path to the real geometry ? 3D vision guided full automated laser cladding repair of complex metallic parts ? Automatic generation of robot trajectories 1. Part scanning and filtering 2. Surface selection and path generation 3. Repair job generation and supervision 3D Filtering Initialization (setup) Scan layer Depth mapTarget Depth map Disparity Data Layer path planning Layer Path Planning (geometrical control) Laser Cell supervisor Robotized Cell 0 Finished Repair Job 3D geometrical control Coated surface
  • 12. openlmd.github.io | jorge.rodriguez@aimen.es 12 Automatic coating of surfaces 3D scanning and robot path planning Enabled by the 3D point cloud directly provided by the 3D geometrical monitoring solution 1. Workarea scanning (direct part information in 1. cell coordinates) [mm] 2. 3D point cloud projection (2D Zmap image) 3. Surface selection directly in the 2D image 4. Segmentation of the Zmap image 5. Contours calculation from the segmented surface 6. Contours and Zmap feed the path planner 7. A new path is automatically calculated from that 1. information A second scanning after the process enables an adaptive path planning strategy in a full automatic way Surface selection 3D scanned surfaces
  • 13. openlmd.github.io | jorge.rodriguez@aimen.es 13 Conclusions ? Integration and data acquisition ? Spatial reference system and temporally synchronized ? Monitoring and control ? Real-time closed-loop laser power control ? Adaptive path planning Work in-progress ? Embedded image monitoring and control ? Big data and deep learning approaches Robot ? Pose ? Process speed 3D geometry ? Point cloud (<0.5mm) SWIR/MWIR-NIR ? 2D melt pool geometry ? Thermal distribution and texture Conclusions and future work Reconfigurable Modular and reconfigurable Interoperability Large parts Low-cost solution Scalability
  • 14. AIMEN ¨C Central y Laboratorios c/ Relva 27 A 36410 ¨C O PORRI?O (Pontevedra) Telf.+34 986 344 000 ¨C Fax. +34 986 337 302 Thank you for your attention Jorge Rodr¨ªguez-Ara¨²jo | Research Engineer Ph +34 986 344 000 | jorge.rodriguez@aimen.es www.aimen.es | aimen@aimen.es This work has receive funding from the European Union¡¯s Horizon 2020 research and innovation programme under grant agreement N? 637081. The dissemination of results herein reflects only the author¡¯s view and the Commission is not responsible for any use that may be made of the information it contains. http://openlmd.github.io

Editor's Notes