Envisioning an Infrastructure for Multi-Robot and Collaborative Autonomy Testing and Evaluation
Recent years have seen significant progress made in standardizing robot software, thanks to ROS and its associated simulation tools such as Gazebo. This has enabled researchers to build on each other’s work without having to reinvent the wheel. Nevertheless, there is a critical lack of infrastructure when it comes to multi-robot systems and collaborative autonomy. The key question that we would like to discuss at this workshop is: what should the testing and validation infrastructure for multi-robot and collaborative autonomy look like.
Researchers who are inventing and enhancing new collaboration algorithms sometimes resort to testing their innovations on a small set of simplified scenarios that they have used in their previous work in order to reduce the time and cost of testing. Such testing may be merely a demonstration that the collaboration algorithm works; it may not yield comparisons with existing algorithms that generate insights into the performance and cost tradeoffs; and it may not consider a broad enough set of missions and scenarios needed for test and evaluation, validation and verification (TEVV). The multi-robot systems community needs an infrastructure that supports system development and testing from concept to realization in multiple domains. The goal of this workshop is to bring together researchers and practitioners to understand the infrastructure needs of collaborative autonomy testing.
The University of Maryland recently received a planning grant from the CISE Community Research Infrastructure (CCRI) program to envision what an Infrastructure for Collaborative Autonomy Testing (ICAT) would look like. Through this workshop, we seek to foster a discussion with the broader community, understand the needs of the stakeholders, and identify effective resources that might be incorporated into ICAT. Through this workshop, a larger portion of the CISE MRS research community will be able to participate in the design of ICAT.
University of Virginia
University of Bristol
TU Berlin
Purdue University
Open Robotics
Open Robotics
We invite participants to submit their research in RSS format (up to 4 pages including figures, excluding references). We encourage the submission of early ideas, late-breaking results, position papers, or open research questions that are likely to generate interesting discussions. Accepted papers will be presented in a poster session and selected papers as spotlight talks. All submitted contributions will go through a single blind review process.
The specific goals of the workshop will be to discuss ideas around following topics:
Please email submissions (in PDF format) to: rss22eimrcate@gmail.com
RSS Submission Template: https://roboticsconference.org/information/authorinfo/
For more information, see https://raaslab.org/rss2022/
In-person Venue: 1024 Mudd (Morningside Campus, Columbia University)
Remote Participation Link: Zoom link
Time | Event |
---|---|
9:00 | Opening Remarks |
9:15 | Invited Talk - Dr. Nina Mahmoudian (Purdue University) |
10:00 | Coffee Break |
10:30 | Invited Talk - Dr. Morgan Quigley and Dr. Michael Grey (Open Robotics) |
11:15 | Contributed presentations |
12:00 | Lunch Break |
13:30 | Invited Talk - Prof. Sebastian Elbaum (University of Virginia) |
14:15 | Invited Talk - Dr. Wolfgang Hönig (TU Berlin) |
15:00 | Coffee Break |
15:30 | Invited Talk - Dr. Simon Jones (University of Bristol) |
16:15 | Panel Discussion |
17:00 | Closing Remarks |
Invited Talk 1 (July 1, 9:15) | |
---|---|
Low-cost maritime fleet planning and operation Nina Mahmoudian Abstract: This talk will present a holistic approach to low-cost fleet planning and operation for a variety of applications such as physical oceanography, Arctic observation, coastal surveillance, and target detection/classification that depend on the persistent operation of unmanned systems in maritime environments. We accomplish this through addressing the fleet planning problem of diverse fleets along with designing, simulating and validating custom and off the shelf Unmanned Surface Vehicles (USVs), Autonomous Underwater Vehicles (AUVs), and Underwater Gliders (UGs). In this talk, I will describe recent progress on mission planning considering realistic constraints on performance and degradation, design and operation of a custom small underwater glider ROUGHIE, development, optimization, testing, and validation of a collapsible underwater docking station for a range of AUVS, development and testing of a custom low cost USV, and validation of mission planning maneuvers using off the shelf AUVs. Simultaneous validation of multiple vehicles is achieved through surrogate models such as a fleet of low-cost, compact aerial platforms. This approach enables a lower cost and faster transition of novel persistent fleet technology to the field compared to directly transitioning from simulation to multiple high-cost assets. Bio: Dr. Nina Mahmoudian is an associate professor of Mechanical Engineering at Purdue University. Previously, she was the Lou and Herbert Wacker Associate Professor in Autonomous Mobile Systems with the Department of Mechanical Engineering at Michigan Tech. She received her Ph.D. in Aerospace Engineering from Virginia Tech. Dr. Mahmoudian is a recipient of the 2015 NSF-CAREER and 2015 ONR-YIP awards. She is serving as editor of ICRA and associate editor of RA-L. |
Invited Talk 2 (July 1, 10:30) | |
---|---|
On simulating multi-fleet operations with Gazebo and Open-RMF Morgan Quigley and Michael Grey Abstract: In recent years, many applications of mobile robots have become commercially viable. Large buildings in a variety of industries now operate fleets of robots for cleaning, delivery, security patrols, and so on. Some buildings are now starting to operate multiple fleets of robots from multiple vendors, and discovering the challenges inherent in multi-vendor operations. In particular, fleets are often unaware of the other fleets in the facility, and although intra-fleet deconfliction is a common feature, inter-fleet deconfliction is often achieved operationally only through spatial or temporal isolation. Such isolation techniques can work up to a point, but often fail to scale beyond two or three fleets. In this talk, we will present Open-RMF, an open-source multi-fleet deconfliction and operations framework. We will describe our Gazebo-based development and testing process. We will also discuss the challenges of "emulating" the behavior of proprietary fleets, and share some thoughts on how simulation testing could be improved in the future, potentially by using vendor-provided fleet management software "in the loop" to reduce surprises in real-world testing. Bio:
|
Contributed Presentations | |
---|---|
11:15-11:22 | Apurva Badithela, Josefine Graebener, Richard M. Murray. Minimally Constrained Testing for Autonomy with Temporal Logic Specifications. |
11:22-11:29 | Micah Corah and Sebastian Scherer. On Performance Impacts of Coordination via Submodular Maximization for Multi-Robot Perception Planning and the Dynamics of Target Coverage and Cinematography. |
11:29-11:36 | Mickey Li, Robert Clarke, Arthur Richards. Starling: Containerisation Architecture for Scalable Local Development, Deployment and Testing of Multi-UAV Systems. |
11:36-11:43 | Vasu Sharma, Prasoon Goyal, Kaixiang Lin, Govind Thattai, Qiaozi Gao, Gaurav S. Sukhatme. CHMARL: A Multimodal Benchmark for Cooperative, Heterogeneous Multi-Agent Reinforcement Learning. |
11:43-11:50 | Georgios Tzoumas, Lucio Salinas, Lenka Pitonakova, Alex Horlock, Jonathon Waters and Sabine Hauert. Developing digital twins of swarms of UAVs for wildfire monitoring. |
11:50-12:00 | Q&A session |
Invited Talk 3 (July 1, 13:30) | |
---|---|
Emerging Testing Techniques to Validate Robot Systems Sebastian Elbaum Abstract: The soaring number of autonomous systems are bringing to prominence the insidiousness and potential impact of their faults. In turn, these faults highlight the limitations of existing analysis techniques, which are being challenged by robots’ immense and rich input space, the integration of cyber and physical semantics, the multiple sources of uncertainty, and the incorporation of machine learning components. In this talk I will provide an overview of some mobile robotic platforms and frameworks that we have targeted to ground our research efforts in addressing these challenges, the range of properties that those systems are meant to satisfy from robustness to ethical concerns, and the body of techniques we have developed to address those challenges. I will argue that such techniques must be an integral part of any testing infrastructure aiming at validating robots. Bio: Dr. Elbaum is Anita Jones Professor of Computer Science at the University of Virginia, and one of the founders of the Laboratory for Engineering Safe Software (LESS Lab). His research aims to build dependable systems through domain-specific analysis techniques. His teaching focuses on instilling cost-effective software development principles. He is the recipient of an NSF Career Award, an IBM Innovation Award, a Google Faculty Research Award, an FSE Test of Time Award 2018, several ACM SigSoft Distinguished Paper Awards (FSE2006, ICSE2008, ICSE2012, ISSTA2013, ICSE2016), several best paper and distinguished artifacts awards (ESEM2011, IROS2016, ISSTA2017, ISSTA2020), mostly for empirically studying program analysis and software testing challenges, and developing automated techniques for addressing them. His current development and analysis work is focused on autonomous systems. He served as Program Co-Chair of the 2015 International Conference on Software Engineering, Program Chair for the 2007 International Symposium on Software Testing and Analysis, Program Co-Chair for the 2008 Empirical Software Engineering Symposium, Co-Editor for the Information and Software Technology Journal, and as Associate Editor of the ACM Transactions on Software Engineering and Methodologies Journal. He was the Steering Committee Chair for ICSE from 2015-2019. He was recognized as a Distinguished Scientist by the Association for Computing Machinery for his contributions to computing, and as a Fellow by the IEEE for his contributions to testing evolving systems. He received his Ph.D. from the University of Idaho, and a Systems Engineering degree from Universidad Catolica de Cordoba, Argentina. He spent a significant part of his academic career at the University of Nebraska, where he co-founded two internationally recognized labs, the E2 Software Engineering Lab and the Nimbus Robotics Lab. He has spent his sabbaticals as a research scientist or research fellow at Google (Mountain View, USA), CNR (Pisa, Italy), and UCL (London, UK). |
Invited Talk 4 (July 1, 14:15) | |
---|---|
Crazyswarm(2): A Testbed for Aerial Robot Teams Wolfgang Hönig Abstract: I will present key concepts and use-cases for the Crazyswarm, a ROS-based test-bed for operating large aerial multi-robot teams that use the commercially available Bitcraze Crazyflie as autopilot. A particular focus will be on how we solved challenges around localization, communication, and simulation, which are often an issue for any multi-robot system. I will also provide an overview of the current state of Crazyswarm2, our re-designed, ROS2-based version of the testbed. Finally, I will present examples and my future vision of how Crazyswarm(2) has been and will be useful for testing and validation of autonomous multi-robot teams. Bio: Wolfgang Hönig is an independent junior research group leader at TU Berlin, Germany, heading the Intelligent Multi-Robot Coordination Lab. Previously, he was a postdoctoral scholar at the California Institute of Technology, USA. He received the diploma in Computer Science from TU Dresden, Germany in 2012, and the M.S. and Ph.D. degrees from the University of Southern California (USC), USA in 2016 and 2019, respectively. His research focuses on enabling large teams of physical robots to collaboratively solve real-world tasks, using tools from informed search, optimization, and machine learning. |
Invited Talk 1 (July 1, 15:30) | |
---|---|
DOTS: An Open Testbed for Industrial Swarm Robotic Solutions Simon Jones Abstract: We present DOTS, a new open access testbed for industrial swarm robotics experimentation. It consists of 20+ fast agile robots with high sensing and computational performance, and real-world payload capability. They are housed in an arena equipped with private 5G, motion capture, multiple cameras, and are openly accessible via an online portal. Our aim is to reduce barriers to entry by providing a complete platform-agnostic pipeline to develop, simulate, and deploy experimental applications to the swarm. As a proof-of-concept, we show how the testbed can be used to design new intralogistics solutions using out-of-the-box swarms. Bio: Simon is a Senior Researcher and Doctoral Research Prize Fellow in the Hauert Lab and Bristol Robotics Laboratory where he works on distributed situational awareness for robot swarms. He's been Co-I and lead researcher on several grants related to building a state-of-the-art swarm testbed for intralogistics in collaboration with industry. He is also researching the automatic design of controllers for robot swarms that are human readable using artificial evolution of behaviour trees, including onboard custom built swarms with GPUs. Prior to academia, Simon was principal engineer at ARM. |
University of Maryland
Applied Research Laboratory
for Intelligence and Security
University of Maryland
University of Maryland
University of Maryland
University of Maryland
University of Maryland