iPhones to lend a hand determine offside in English football this season

The UK’s top football league is changing the technology it uses to detect offside on the pitch. The English Premier League (EPL) has signed a deal with Genius Sports, which will apply dozens of iPhones combined with machine learning models to lend a hand referees make offside decisions.

Offside violation aren’t always clear-cut, especially when players are grouped in a way that prevents referees, or even multiple cameras, from seeing enough detail to pick them out accurately. This is where video assistant referee (VAR) systems typically come in, supposedly filling in the gaps using cameras and machine learning.

Soccer leagues have been using VAR systems for years. FIFA, the global soccer league, officially began using machine-learning-based limb-tracking technology and soccer’s built-in sensors in 2022 after a trial the previous year. The software could track 29 points on players’ bodies, but the systems have limitations and often cause “extensive delays and human process errors” and “concerns about the accuracy of in-game decisions,” I’m writing Wire in the report on the EPL deal with Genius Sports.

Genius calls its offside detection technology “semi-assisted offside technology” (SAOT), said Chief Product Officer Matt Fleckenstein Edge in an interview. It’s part of the company’s GeniusIQ system, which also powers its fan-facing offerings, which create features like lively, real-time graphics (think tracks following a soccer ball).

Genius claims its SAOT technology can accurately create 3D renders of each playerand that it helps referees determine exactly where the offside line is on the pitch and where all the players are in relation to it. To do that, the company needs a lot of cameras.

“We were moving away from 4K cameras, which were much more expensive,” Fleckenstein said. “We wanted to see if we could move to a more accessible mobile phone.” The company eventually moved to iPhones, largely because that was what the company’s employees were most familiar with when it came to things like software development.

Fleckenstein said that “the key is to spread out the 24- to 28-inch iPhones — mostly iPhone 15 Pros — to get even coverage of the pitch and sidelines, usually in pairs of custom setups that each hold two phones at a time and are angled slightly differently to make sure that area is covered.”

This approach apparently gives Genius “between 7,000 and 10,000” data points, which allow it to generate a sort of virtual 3D mesh of each player. Having that many data points means the system can tolerate missing details from things like lighting issues, Fleckenstein said.

In addition, iPhones can record at very high frame rates—Genius records at 100 fps, but it tested up to 200 fps—and the phones also offer local computer vision processing. All data is sent to a local server, where it is processed by the GeniusIQ system.

GeniusIQ’s computer vision and predictive algorithms crunch the data to identify individual body parts—right down to a player’s fingertips—and predict where they are when they’re out of sight. The company trained its system on “several seasons” of soccer matches to do this, according to Wire.

All of this is done to determine where each player is in relation to the others, the ball and the goalkeeper. Fleckenstein said an offside is called “when the ball leaves the attacker’s foot,” so having more frames recorded increases the likelihood that the cameras will capture the exact moment of the event.

The official rules of soccer are very specific about what makes a player “offside,” but it’s unclear whether the greater detail makes GeniusIQ any better than existing alternatives. Fleckenstein didn’t provide any performance comparisons, but he noted that other VAR systems might only apply “30 or 40 body points,” creating a sort of rugged version of a stick figure player. Or they might only apply “center of mass” tracking, where each player is represented by a single data point.

We’ll soon find out if Genius Sports’ offside detection technology can really do a better job than the VAR systems of the past few years. It’s expected to be fully deployed by the EPL before the end of the year and continue throughout the season. Fleckenstein said an exact date hasn’t been announced.