Rapyuta Robotics

In October 2018, a cloud robotics company from Japan approached us to help design their autonomous drone platform for enterprises

MY ROLE

I headed the UX and UI design for the revamp of their existing platform. Working closely with their Product Team we created the next version of their io drone mobile app as well as a web app with advanced features.

BACKGROUND

Rapyuta Robotics' mission is to make robotics more accessible to new markets. They have created easy to use robotics platforms using the power of cloud technologies. One of their major offerings is their io.drone platform, which can be used to create drone missions and execute them, even if the users are novice pilots.

io.drone platform helps users in industries like construction, fishing, agriculture, energy, security and entertainment to survey an area and generate relevant data to make necessary decisions quickly.

EXISTING USER FLOW

We started the design process by inspecting the existing user flow of the web app. We explored the product in detail and mapped it to identify the problems in the existing flow.

USER INSIGHTS

The research phase of the project started with user interviews. We interviewed some prospective customers in India and also the customer service reps in Japan to gain an understanding of how the users feel about the existing product.

The insights we received were analyzed and we plotted them on a customer journey map during a product management meeting. This helped us in understanding the problem in detail by talking about them with the product management. By putting the major insights on a customer journey map, we were able to quantify the problems to an extent.

HEURISTIC EVALUATION

Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
Observation: The forms appearing in the website need appropriate labels to help the user understand their functions. The UI elements should chnage state or show more information(tootip) on mouse hover. It is hard to differentiate between cards and buttons now.
Match between system and the real world
The system should speak the user's' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
Observation: Information is presented in a logical order.
User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Observation: View Flight Button is confusing
Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
Recognition rather than recall
Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Flexibility and efficiency of use
Accelerators — unseen by the novice user — may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
Aesthetic and minimalist design
Dialogues should not contain information that is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Observation: Can be improved
Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
Observation: - There is no proper support for different types of information (Videos, Images, FAQs)

THE VISION

Revamp the entire web UX
The client felt that the existing design is outdated and more importantly isn't scalable. They were planning to introduce a bunch of new features and the ability to incorporate 3rd part apps. The new design should consider these changes and should be scalable for the next 12 - 24 months.
New features
REVISIONS/PRESETS - The missions happening at the same location but slightly different boundary should be able to create and keep track of easily. Also, missions with the same boundary but executed using different parameters should be clubbed together.

VERTEX ACTIONS - The user should be able to assign different actions for the drone at different vertices of the boundary/path. Ths means, in the same mission, the user can manoeuvre the drone in different ways and can capture data in different modes.

MISSION PLANNING - Add more options to mission planning to help make the missions errorless as possible. This includes the ability to define Safe and Danger zones along a flight path to let the system know what to avoid and where to crash if the drone faces mid-flight problems.
Telepresence
The ability to keep track of ongoing missions using the web interface. The admin should be able to see the live stream of missions and monitor the data.
Teleoperation
The ability to run missions remotely using active drones. Also, the admin can take manual control of any ongoing mission and fly the drone around using the web interface.

COMPETITORS

DroneDeploy

Cloud software platform for drone mapping and 3D modelling. App provides easily automated flight and data capture and enables you to explore and share high-quality interactive maps in the field directly from your mobile device. Users can process the imagery with DroneDeploy to generate high-resolution maps and 3D models, then analyze using 30+ tools available for install within the DroneDeploy interface from the App Market.

USER FLOW:

Fly & Capture Images ---> Map & 3D Models ---> Explore,Analyse & Share

The mobile app allows users access to automated mapping services which are suitable for both beginners and professionals.

The app makes it simple to create flight plans on any device allowing you to launch a mission with automated take-off, flight, image capture and landing.

You can keep on top of your flight through a live first-person view and are free to take back control of the drone at any time.

App Features:

Intuitive user experience packed with functionality
Instantly measure both area and volume
Access your orthomosaics, NDVI data, interactive maps and 3D models
Collaboration is made easy via shared maps and comment functionality

Pix4D

Pix4Dcapture turns your consumer drone into a professional drone mapping tool. Is the perfect tool to automatically capture image data - RGB, thermal - for optimal 3D models and maps. Process post-flight images easily on the cloud or desktop applications, producing georeferenced maps and models that are tailored to many industry needs. The photogrammetry aspect of the software uses the images captured by UAVs to generate results ranging from point clouds to digital surface and terrain models, orthomosaics to textured models.

App Feature:

Create 3D Point Clouds: Offering a series of results that are ‘denser’ than traditional laser-scanning, the Pix4D point cloud is derived from a number of overlapping images which gives you precise locational data on your reconstruction of the object space
Contour Lines: For a quick, simplified visualisation of an area’s topography, this feature lays out a series of contours which roughly display the elevation of your subject.
Cloud-Based Timeline: As you collect your data it is displayed on an intuitive timeline broadcast over the cloud and allows 2D and 3D data to be accessed, as well as annotated and shared.

BIM Comparison: You’re able to import drawings into the Pix4Dbim cloud to compare ‘as-built’ orthophotos with ‘as-design’ concepts, which means that construction errors can be spotted before they have a chance to impact the build and schedule diagrams ensure that you’re on track to hit targets.

Application Maps: Collate and visualise the data from your index maps and export them as a shapefile (*.shp) which can be directly imported into farm equipment for application in the field

Forensics: It’s crucial to get as much detailed information as possible from a crime scene. Pix4D’s aerial imaging solution gathers this critical data rapidly and effectively allowing for the preservation of visual evidence

WEB APP -
VERTEX ACTIONS

INFORMATION ARCHITECTURE

Since the product was undergoing a major overhaul the existing Information Architecture and Navigation Flow had to be completely updated. We created the IA for both the web and mobile apps based on our findings from the Discover Phase.

MOBILE APP - WIREFRAMES & NAVIGATION FLOWS

The mobile app was designed primarily for the Pilots, who will be executing the missions on site. The main objective of the app is to provide the Pilots information regarding a mission and help them connect their mobile device to the Drones so as to execute the mission.

All the major functions were divided into 4 categories, which made the 4 primary navigations for the application.

MOBILE APP -
USER INTERFACE

A minimal UI was created for the mobile app, based on the wireframes and navigation flow. We followed one-to-one style of design which will help the clients use the same design(with minor changes) for both Android and iOS platforms.

The clients wanted a simple and minimal interface for this product. After much deliberation and iterations, we decided to adopt a design language that is greatly inspired by the new Google design language.

Here are some of the screens that I am free to share here.

One of the major additions to the new app is the ability to monitor and control live missions through a web interface.

The admin can sit anywhere in the world and can see the live feed from the drones which are flying at a time. The admin also has the ability to take manual control of these drones and fly them around using the keyboard and mouse.

Another great feature is to let the admin take any of the available drones and fly that around remotely. The available drones are visible to the admin and they can choose to fly them whenever they want.

WEB APP -
LIVE OPERATIONS

The new application should be able to run complex missions. In this case, a mission can be a combination of paths and areas and also the media mode for each segment can be different.

To let the users customize their missions to this extent, we introduced 'Vertex Actions'. The user will be able to designate actions to a vertex.

So, if a mission starts from vertex H, the user can set the action as Photo and the drone will take photos for the rest of the mission.

But if the user wants to take Orthomap of the area starting from vertex V, they include an action for that vertex with the specifications. The drone will start taking an orthomap as it reaches V.

I the users want the drone to continue taking from V, once it has completed the orthomap, the user can include a second action for that vertex. This action will take place when the drone reaches that vertex for the second time.

WEB APP -
MISSIONS

The missions user flow is the most complicated part of this product. This primarily included creating a new mission and there are multiple steps involved in that process.

The user can create a mission from the Missions landing page. They can either create a new mission or can also duplicate an existing mission, in case they want to utilize any of the settings used in another mission.
Creating a new mission starts with a pop-up to name the mission and also to select the type of the mission. The type of mission is selected at this stage to help customize the edit interface with tools that are relevant to a mission.
The user is taken to a map-based interface, where they search for a location and start drawing the mission path, using the drawing tools provided on the left sidebar. The user can create a flight path and can also denote other specific sections to help the mission run smoothly.
A danger zone can be drawn within the mission are which denotes the are that needs to be avoided when the drone is executing that mission.
A safe zone can be drawn, which denotes a safe area for the drone to land. This is particularly useful when the drone is covering large areas. In case the drone encounters any technical difficulty, it'll land in the safe area and that makes it easy for the pilot to locate the drone.
Once the mission path is drawn, the actions can be assigned to the path. This can be done with the help of vertex actions which is explained in the previous section.
A mission can be assigned to a Pilot. The admin can also set timings for the mission. There are options to set recurring mission timings, which helps the system provide notifications to the Pilot.