Founded in 2003 as Tesla Motors, the electric vehicle and clean energy company based in California currently has one Market capitalization of over 700 billion US dollars – and therefore more valuable than the seven largest automakers combined. Today Tesla is known for its electric vehicles, but also produces products for consistent Power generation and storage such as solar panels, solar roof tiles and more to “enable homeowners, businesses and utilities to manage the generation, storage and consumption of renewable energy”.
Tesla claims its mission is to “accelerate the world’s transition to sustainable energy,” and in 2020 the company sold more than 500,000 units its electric cars produced more than 1 million vehicles. In the same year, Tesla also had the highest sales figures in the plug-in and battery-electric passenger car segments.
In this article, we examine two different AI and ML-related use cases at Tesla:
- Computer vision for autopilot AI – How Tesla is partnering with PyTorch to create, train, and improve the neural networks that aim to enable its cars’ autopilot technology
- Powerhub energy asset monitoring – Powerhub is one of the software developed by subsidiary Tesla Energy as part of the company’s suite of energy optimization, management and storage solutions called Autonomous Control
Computer vision for autopilot AI
Tesla claims that autopilot is a driver assistance system that aims to improve the safety and comfort of the driver. If used correctly, the system could potentially reduce the overall effort of the driver. Computer vision is only part of a wider range of AI applications that enable autopilot. Autopilot allows users the convenience of driving themselves and allegedly more safety.
The company offers five different models and each one new Tesla vehicle According to the company, it is equipped with “8 external cameras, 12 ultrasonic sensors and a powerful on-board computer” as well as autopilot technology. Autopilot relies on these cameras and sensors as the company doesn’t use light detection and ranging (LiDAR) or high-resolution maps to interact with the environment like its competitors.
One possible reason Tesla is the only major automaker to focus on computer vision rather than LiDAR could be data diffusion as the company collects all of its vehicles’ data to further train and improve its deep neural networks . Andrei Karpathy, Head of AI and Computer Vision at Tesla, explains below a conference in 2019 that Tesla is generally a vertically integrated company, and it is no different with autopilot intelligence.
Karpathy went on to explain that the company builds its own vehicles, puts sensors on each vehicle, collects all of its own data, labels and uses that data to “train GPU clusters and then run them through the whole stack, we run those networks.” to “our own custom hardware that we develop in-house and then … we put it in our fleet of nearly three quarters of a million cars and look at the telemetry and try to improve it[s] over time.”
An example of the video feed analysis and object identification of a vehicle. Source: Tesla
He explains that his team worked with PyTorch to train multi-headed neural networks, or “hydrranets”, to analyze and collect data (images) for road boundaries, traffic lights, obstacles, cars, and other things they can interact with. He explains how the neural networks output a street layout prediction:
You very quickly come across tasks that have to be a function of several images at the same time. For example, if you are trying to predict street layout, you may need to borrow features from several other fire hydrants. So it looks like we have all these different hydrants for different cameras, but then you might want to pull some functionality out of those hydrants and go through a second round of processing, optionally recurring, and actually doing something like a street layout prediction.
According to Tesla, there are a number of features that can be used with Autopilot technology, including Autopilot Navigation, Autosteer +, Smart Summon, Autopark, and more. The company demonstrates here how to safely activate these functions while driving on the motorway:
According to Teslas Q4 2020 security report Accident data recorded only one accident per 3.45 million kilometers driven when users activated the autopilot. The company further stated,
“For those who drive without an autopilot but with our active safety functions, we recorded one accident for every 2.05 million miles driven. For those who drive without an autopilot and without our active safety functions, we have recorded one accident for every 1.27 million miles driven. By comparison, the latest data from the NHTSA shows that there is a car accident every 484,000 miles in the United States. “
Powerhub energy asset monitoring
Autonomous Control is the name of Tesla’s software suite that the company claims uses machine learning, forecasting, optimization and other technologies to potentially benefit its users in a variety of ways, from reducing energy costs to facilitating remote microgrid control . This software suite contains four different programs that aim to achieve their stated goals: Autobidder, Powerhub, Microgrid Controller and Opticaster.
Powerhub is described by the company as an “advanced monitoring and control platform” that is intended to be used to manage individual or company-owned energy systems and resources. The software platform displays critical information from various energy systems on a single interface, including “data for site miners, individual battery blocks, solar inverters and diesel generators”, which is intended to provide a comprehensive overview of these important key figures.
An example of the Powerhub dashboard data analysis. Source: Tesla
Colin Breck, Senior Staff Software Engineer and Cloud Platforms Lead at Tesla, speaking at the QCon Software Development Conference, noting that the Tesla energy platform is designed for both residential and industrial consumers:
For private customers, the platform supports products such as the Powerwall home battery, which can provide emergency power for a house for hours or days in the event of a power failure, the solar roof, which generates electricity from beautiful roof tiles, and the retrofitting of solar … We use software to To provide an integrated product experience for solar generation, energy storage, emergency power supply, transportation and charging of vehicles. Part of the customer experience is seeing the real-time performance of this system on the mobile app and customers can control some behaviors such as prioritizing the store during times of low cost.
To date, we have not been able to identify any specific use cases for customers with reported improvements to their operations due to Powerhub or other autonomous control software and products.
However, the company gave up in his Impact report 2020: “By the end of 2020, Tesla (including SolarCity before the takeover by Tesla in 2016) had installed almost 4.0 gigawatts of solar systems and accumulated over 20.8 terawatt hours (TWh) of emission-free electricity.”
Tesla claims these numbers generated many times more clean energy than the company needed to run all of its vehicle factories since production began in 2012. They presented these key figures in a bar chart:
Bar graph of the total energy produced by Tesla solar panels versus the energy used in its factories.
Finally, the company reported that in 2020 all of its products, from vehicles to solar panels, will enable Tesla customers and customers to avoid 5.0 million tons of carbon dioxide emissions.