01Jetson Orin NX / Orin Nano / NX 4G-8G
Omnidirectional platform
COLL/CSCI 100 Physical Intelligence:
Robots
How do intelligent robots perceive the world, know where they are, and decide what to do next?
In this hands-on course, students work with 5G-connected robotic vehicles on a reconfigurable roadway with programmable smart traffic lights in the Vehicle Computing Lab.
Experiential Learning
Perception / Planning / Control
Learning Cycle:
-
Plan
Ask a focused question and predict what the robot will do.
-
Build and Test
Use provided hardware, starter code, and visual tools to configure and test a behavior.
-
Show
Present a short team demonstration.
-
Explain and Reflect
Use evidence to explain results, including surprises, limitations, and improvements.
Five Course Milestones
Milestone 01
Design of a Smart City Scenario in CARLA Simulator
Students use a preconfigured CARLA simulator to create or modify a city scenario by setting routes, traffic, vehicles, pedestrians, obstacles, roadside conditions, and weather. Teams may introduce challenging situations such as congestion, blocked routes, near-conflicts, or simulated collisions to observe autonomous-vehicle responses.
Expected outcome: Each team presents its customized smart-city simulation, showing the scenario, vehicle behavior, initial prediction, and observed results.
Milestone 02
Turn a Campus Drive into Digital Data
Students take turns teleoperating a large robotic vehicle along a designated campus route. The vehicle collects camera and LiDAR data as it moves through the physical environment. Students learn how data are transferred from the robot, organized, and viewed. They examine two-dimensional camera images, view three-dimensional LiDAR visualizations, and observe pretrained AI models applied to the collected data, such as object detection.
Milestones 03–05
Coming soon — the remaining milestones are in progress this semester.
Research platforms
Our robots for education.
02Jetson Orin Nano / Orin NX
Ackermann steering
ROSMASTER R2 (x2)
Research vehicle supporting perception, planning, and control experiments. View platform details03Vehicle platform
Autonomous vehicle
Rova-M (x2)
Research vehicle supporting perception, planning, and control experiments.04Vehicle platform
Autonomous vehicle
RobiX Robot (RovaX)
Research vehicle supporting perception, planning, and control experiments.05RSU
RSU
Roadside Unit
Research vehicle supporting perception, planning, and control experiments.06Workstation
GPU
Alienware Aurora RTX 5090
Research vehicle supporting perception, planning, and control experiments.07Workstation
GPU
Alienware Aurora RTX 2080 Ti
Research vehicle supporting perception, planning, and control experiments.08Vehicle control
Simulator workstation
Logitech G920 Controller
Research vehicle supporting perception, planning, and control experiments.09Edge server
Simulation Environment
CARLA Simulator
Research vehicle supporting perception, planning, and control experiments.10Communication
Network
Wi-Fi 7 Infrastructure
Research vehicle supporting perception, planning, and control experiments.11Road Regulation
Traffic Management
Traffic Signs
Research vehicle supporting perception, planning, and control experiments.12AI processing
Computing power
NVIDIA DRIVE Thor (×2)
Research vehicle supporting perception, planning, and control experiments.Now in the lab
Explore Undergraduate Research
Lane detection test
Camera view of vehicle
Milestone 1 Active
Autonomous Vehicle in the CARLA Simulation Environment
This experiment uses CARLA to evaluate autonomous driving algorithms in realistic and repeatable traffic scenarios. The simulated vehicle combines perception, route planning, and control to follow lanes, avoid obstacles, and respond safely to changing road and traffic conditions.
Obstacle detection test
Camera view of vehicle
Milestone 2 Active
Obstacle Detection and Tracking
The system detects and tracks nearby vehicles, pedestrians, and other obstacles in real time using onboard camera data. Continuous position and motion estimates help the autonomous vehicle maintain a safe path and respond quickly to changing surroundings.
Lane detection test
Camera view of vehicle
Milestone 3 Active
Autonomous Vehicle Lane Detection and Route Planning
Camera-based perception detects lane boundaries and road markings in real time, enabling the vehicle to plan safe routes, follow lanes, and react accurately.
Connected testbed
Highly interactive course
We instrument vehicles, roads, networks, and edge computers together, so a systems idea can be measured end to end.
01Vehicle layerRGB cameras · IMU · LiDAR · CAN
02Roadside layer4 RGB cameras · 3D LiDAR · RSU
03Network layer5G NR · Wi-Fi 7 · V2X
04Compute layer Alienware Aurora Workstations
Urban Map6 m × 7 mlanes · crosswalks · parking
Course instructor
Meet the instructor.
Assistant Professor · Computer Science
Sidi Lu
Pronounced “see-dee loo”
Dr. Lu’s research focuses on cyber-physical and intelligent systems, with particular interests in edge computing, autonomous systems, and vehicle computing. Her work develops reliable, scalable, secure, and efficient computing systems for connected vehicles and robots. These technologies enable intelligent decision-making and support safe, responsive, and dependable operation across resource-constrained platforms in dynamic, real-world environments.
- sidi@wm.edu
- Office
- ISC 1382
- Department
- Computer Science, William & Mary
Guest lecture
Guest Lecturer Spotlight.
Ph.D. Researcher Johora Akter Polin
Graduate Researcher · William & Mary
Research Interests
Johora Akter Polin is a Ph.D. student at William & Mary whose research focuses on edge intelligence and dependable connected and autonomous vehicles. Her recent work explores efficient edge AI, safety-aware vehicle control, and VLM-enhanced perception for reliable autonomous driving.