Measurement Lab Pulls Out of FCC’s Mobile Broadband Measurement Program
Measurement Lab is pulling out of the FCC’s mobile broadband measurement program, citing irreconcilable differences on how best to collect data in the mobile space. “We don’t feel that this technology, and that the measurement, is ready at this point to produce the kind of research-quality data that we need to support our program,” Meredith Whittaker of Google Research told a group of industry, governmental and academic participants at a meeting Wednesday. M-Lab will have to “back out of the mobile measurement process at this stage, in this form,” she said. M-Lab will still be involved in the FCC’s wireline testing program. An FCC spokesman declined to comment.
Sign up for a free preview to unlock the rest of this article
If your job depends on informed compliance, you need International Trade Today. Delivered every business day and available any time online, only International Trade Today helps you stay current on the increasingly complex international trade regulatory environment.
After discussions with various researchers, M-Lab, run by the New America Foundation’s Open Technology Institute, Google and academic researchers, concluded that the FCC is too quickly rushing forward with mobile broadband testing. There are still some “very subtle, deep questions” to be answered, said M-Lab engineer Thomas Gideon from the Open Technology Institute. M-Lab wants to allow time for those questions to “mature and evolve,” so they can “find models that make, in our view, more sense.” The FCC is taking assumptions they have from their wireline measurement experience, and applying those assumptions to mobile even where it doesn’t make much sense, he told us. “It’s important to get this right."
M-Lab’s concerns involve questions of verifiability and transparency, Gideon said. “We understand that there’s state-of-the-art research in terms of crowd blending and other statistical models for preserving anonymity,” he said. M-Lab has no interest in individual location data, which the FCC has clearly stated it doesn’t plan to use. Yet some “raw performance data” could be useful, and it needs to be accessible “with some sense of contextual data that makes it verifiable,” he said. But the FCC keeps talking about “processing and filtering” the data to strip out a lot of data in the name of privacy, in the process conflicting with M-Lab’s stance on “open science,” he said. In the mobile context, there are ways to connect a handset’s IP address with performance data in ways that don’t compromise personally identifiable information, he said. “There was a willingness on their part to pursue something like that up to a point,” Gideon said of the FCC, but “we just couldn’t get to a point where we could agree."
M-Lab, a consortium of industry and research groups, has been involved with the FCC’s measurement programs since their inception in 2010. The FCC has used the group’s server platform to measure broadband data since then, although the relationship has at times been rocky. In March, the FCC decided to restart a month-long data collection after discovering problems with some M-Lab data. After that ISP and M-Lab officials said things started becoming more adversarial (CD Aug 6 p5).
The FCC determined last month that the risks of collecting unique handset identifiers outweighed any potential benefits to its mobile measurement program (CD Jan 7 p8). “From the get-go, there is no personally identifiable information,” said FCC attorney James Miller, who’s heading the privacy aspects of the program: “There is no unique identifier.” Users won’t even require a user name, as the FCC has eliminated the registration requirement. This means “no problems with forgotten passwords, or conflicting email addresses,” said Sam Crawford, president of the U.K.-based SamKnows measurement company, with which the FCC has a contract.
An FCC official told us the commission makes the protection of consumer privacy a priority. The commission hopes to be able to collect data in a statistically valid manner and in open collaboration with industry for the benefit of consumers, the official said. The FCC will continue to work with M-Lab and industry partners, and hopes that their efforts are successful in developing new measurement strategies, the official said.
Crawford presented mockups of the smartphone app at Wednesday’s meeting. He touted the improved testing functionality of the app, which will run tests more quickly than in earlier versions of the software. Users will be able to determine in the settings how much data they want to use. The group is soliciting advice on what to set the default at -- currently it’s set at somewhere less than 300 MB per month. All data will come directly out of a user’s data plan, said Walter Johnston, chief of the Electromagnetic Compatibility Division in the FCC’s Office of Engineering and Technology. It’s a “complex issue” for carriers to “give a credit” or “not charge for that usage,” he said, which is why the app will minimize the number of tests per user, and set an adjustable data threshold that can be consumed per month, he said. “If they want to cut it down to a much smaller number, they can do that,” Johnston said. But based on past experiences, “a lot of people are interested” and “willing to take a small amount of a data plan and allow it to be used for the common good."
M-Lab’s server infrastructure will no longer be used for the mobile measurement project, Gideon said. The FCC project with SamKnows is just one of a dozen experiments that M-Lab’s working on; M-Lab “doesn’t exist solely in support of the FCC,” he said. But M-Lab is “totally open” to coming back to the mobile project if their concerns could be worked out, Gideon said. “We concede this is a difficult question with privacy, hence our desire to spend more time on it.”