TOUCHSCREEN HMI
CONTROLLING THE SWARMS
The near future could see autonomous vehicle
swarms plowing, fertilizing or harvesting in
concert, independent of human drivers. Guiding
and monitoring these robotic machine battalions
will require entirely new human-machine interface
(HMI) forms, no longer limited to a single,
conventional cabin.
“We do HMIs for swarms of three to five
agricultural machines,” says Professor Jens
Krzywinski, head of industrial design and
engineering at the Technical University of Dresden.
“We are conceptualizing and prototyping one such
HMI for a regional cluster including John Deere, to
be presented at Agritechnica 2019.” Based on
cabinless tractors working in a modular framework,
this system will switch adaptively between three
viewing modes, including a bird’s-eye view
providing whole-process oversight. “Overseeing a
swarm you are no longer connected to the
machines by a cabin, so it involves a completely
different operator mindset,” Krzywinski adds.
Gesture-based systems – involving wireless,
handheld joysticks – could help to externalize HMI
functions. “The operator is not optimally positioned
for different tasks in a cabin, so it makes sense to
move the steering outside of the machine,”
Krzywinski explains.
iVTInternational.com June 2019
36
On the Web
Watch the launch video for Liebherr’s INTUSI at
www.iVTinternational.com/intusi
Gesture-based control could be used to teach
the machine simple operations, like unloading the
first bale from a trailer, which are then repeated
autonomously. “You start the procedure, setting the
goals and parameters, then it continues by itself.
Such a tool could fulfil tasks in a different way,
more fluent, more intuitive and easier to learn than
what we do today in a cabin.”
Krzywinski was part of the interdisciplinary
team behind Liebherr’s INTUSI HMI. “We sketched
input has been received. “Haptic
feedback uses real acceleration with
an actuator moving the screen,
telling you, ‘Ah, it has taken my
input!’” Jendis explains. “We
combine that with acoustic feedback,
displaying a click at the right
moment, to give the user experience
of a real switch on the screen.”
Indeed, he is an advocate of
introducing 3D elements to cabin
touchscreens, either by adhering
out several concepts, including completely mobile
solutions, like a gaming console that you take out of
the cabin, as well as the modular, adaptive one
which became INTUSI,” he reveals.
Krzywinski also praises the immediate, practical
benefits promised by the work being carried out
by the HMI Cluster (See Can The HMI Cluster
Standardize Controls?). “It is not about a far future,
but an attempt to make the daily routine easier for
operators today,” he says.
“OVERSEEING A SWARM YOU ARE
NO LONGER CONNECTED BY
A CABIN, SO IT INVOLVES
A COMPLETELY DIFFERENT
OPERATOR MINDSET”
Professor Jens Krzywinski, head of industrial design and engineering,
Technical University of Dresden
components or shaping the glass
surface. “If I can put my hand on
a feelable area and know where I am
on the screen, I can look outside
and operate a screen blindly. That
might be the next area,” he
concludes, “that the display is no
longer a display but an HMI surface
which is flexible in its content.”
Our windows on machine reality
could soon look far more fluid in
their contours. iVT
ABOVE AND RIGHT: In the
future, remotely controlling
multiple vehicles will involve
a completely different
approach to HMI which
could include wireless
joysticks, but will almost
certainly still rely on
touchscreens to some extent
/iVTInternational.com
/intusi