Month: June 2020

On CAPTCHAs and accessibility (part II)

[continued from part I]

Accessibility as a value system

Accessibility has always been part of the design conversation during product development on every MSFT team this blogger worked on. One could cynically attribute this to commercial incentives originating from US government requirement for software compliance with Americans with Disabilities Act. Federal sales are a massive source of Windows revenue and failing a core requirement that would keep the operating system out of that lucrative market is unthinkable. But the commitment to accessibility extended beyond the operating system division. Online services under the MSN umbrella arguably had even greater focus on inclusiveness and making sure all of the web properties would be usable for customers with disabilities. As with all other aspects of software engineering, individual bugs and oversights could happen, but you could count on every team having a program manager with accessibility in their portfolio, responsible for championing these considerations during development.

Luckily it was not particularly difficult to get accessibility right either, at least when designing websites. By the early 2000s, standardization efforts around core web technologies had already laid the foundations with features specifically designed for accessibility. For example, HTML images have an alternative text or alt-text attribute describing that image in words. In situations when users can not see images, a screen-reader software working in conjunction with the web browser could instead speak those words aloud. World Web Consortium had already published guidelines with hints like this— include meaningful alternative text with every image— to educate web developers. MSFT itself had additional internal guidelines for accessibility. For teams operating in the brave new world of “online services” (as distinct from the soon-to-be-antiquated rich-client or shrink-wrap models of delivering software for local installation) accessibility was essentially a solved problem, much like the problem of internationalization or translating software into multiple languages which used to beguile many a software project until ground rules were worked out. As long as you followed certain guidelines— obvious one being to not hard-code English language text intended for users in your code— your software can be easily translated for virtually any market without changing the code. In the same spirit, as long as you followed specific guidelines around designing a website, browsers and screen readers will take care of the rest and make your service accessible to all customers. Unless that is, you went out of your way to introduce a feature that is inaccessible by design— such as visual CAPTCHAs.

Take #2: audio CAPTCHAs

To the extent CAPTCHAs are difficult enough to stop “offensive” software working on behalf of spammers, they also frustrate “honest” software that exists to assist users with disabilities navigate the user interface. Strict interpretation of W3C guidelines dictates that every CAPTCHA image is accompanied by an alternative text along the lines of “this picture contains the distorted sequence of letters X3JRQA.” Of course if we actually did that, spammers could cheat the puzzle, using automated software to learn the solution from the same hint.

The natural fallback was an audio CAPTCHA: instead of recognizing letters in a deliberately distorted image, users would be asked to recognize letters spoken out in a voice recording with deliberate noise added. Once again the trick is knowing exactly how to distort that soundtrack such that humans have an easy time while off-the-shelf voice recognition software stumbles. Once again, Microsoft Research to the rescue. Our colleagues knew that simply adding white-noise (aka Gaussian noise) would not do the trick. Voice recognition had become very good at tuning that out. Instead the difficulty of the audio CAPTCHA would rely on background “babble”— normal conversation sounds layered on top of the soundtrack at slightly lower volume. The perceptual challenge here is similar to carrying on a conversation in a loud space, focusing on the speaker in front of us while tuning out the cacophony of all the other voices echoing around the room.

As with visual CAPTCHAs, there were various knobs for adjusting the difficulty level of the puzzles. Chastened by the weak security configuration on the original rollout, this time more conservative choices were made. We recognized we were dealing with an example of the weakest-link effect: while honest users with accessibility needs are constrained to use the audio CAPTCHA, spammers have their choice of attacking either one. If either option is significantly easier to break, that is the one they are going to target. If it turns out that voice-recognition software could break the audio, it would not matter how good the original CAPTCHA was. All of the previous work optimizing visual CATPCHAs would be undermined as rational spammers shift over to breaking the audio to continue registering bogus accounts.

Fast forward to when the feature rolled out, that dreaded scenario did not come to pass. There was no spike in registrations coming through with audio puzzles. The initial version simply recreated the image puzzle in sound, but later iterations used distinct puzzles. This is important for determining in each case whether someone solved the image or audio version. But even when using the same puzzle, you would expect attackers requesting a large number of audio puzzles if they had an automated break, along with other signals such as a large number of “near misses” where the submitted solution is almost correct except for a letter or two. There was no such peak in the data. Collective sigh of relief all around.

Calibrating the difficulty

Except it turned out the design missed in the opposite direction this time. It is unclear if spammers even bothered attacking the audio CAPTCHA, much less whether they eventually gave up in frustration and violently chucked their copy of Dragon Naturally Speaking voice-recognition software across the room. There is little visibility into how the crooks operate. But one thing became clear over time: our audio CAPTCHA was also too difficult for honest users trying to sign up for accounts.

It’s not that anyone made a conscious decision to ship unsolvable puzzle. On the contrary, deliberate steps were taken to control difficulty. Sound-alike consonants such as “B” and “P” were excluded, since they were considered too difficult to distinguish. This is similar to the visual CAPTCHA avoiding symbols that look identical such as the digit “1” and letter “I,” or the letters “O” and “Q” which are particularly likely to morph into each other as random segments are being added around letters. The problem is all of these intuitions around what qualifies as “right” difficult level were never validated against actual users.

Widespread suspicion existed within the team that we were overdoing it on the difficulty scale. To anyone actually listening to sample audio clips, the letters were incomprehensible. Those of us raising that objection were met with a bit of folk-psychology wisdom: while the puzzles may sound incomprehensible to our untrained ears, users with visual disabilities are likely to have  far more heightened sense of hearing. They would be just fine, this theory went: our subjective evaluation of difficulty is not an accurate gauge because we are not the target audience. That collective delusion may have persisted, were it not for a proper usability study conducted with real users.

Reality check

The wake-up moment occurred in the usability labs on MSFT Redmond-West (“Red-West”) campus. Our usability engineer helped recruited volunteers with specific accessibility requirements involving screen readers. These men and women were sat down in front of a computer to work through a scripted task as members of the Passport team stood helpless, observing behind one-way glass. To control for other accessibility issues that may exist in registration flows, the tasks focused on solving audio CAPTCHAs, stripping away every other extraneous action from the study. Volunteers were simply given dozens of audio CAPTCHA samples calibrated for different settings, some easier and some harder than what we had deployed in production.

After two days, the verdict was in: our audio CAPTCHAs were far more difficult than we realized. Even more instructive were the post-study debriefings. One user said he would likely have asked for help from a relative to complete registering for an account— the worst way to fail customers is making them feel they need help from other people in order to go about their business. Another volunteer wondered aloud if the person designing these audio CAPTCHAs was influenced by John Cage and other avant-garde composers. The folk-psychology theory was bunk: users with visual disabilities were just as frustrated trying to make sense of these these mangled audio as everyone else.

To be clear: this failure rests 100% with the Passport team— not our colleagues in MSFT Research who provided the basic building blocks. If anything, it was an exemplary case of “technology transfer” from research to product: MSR teams carried out innovative work pushing the envelope, problem, handed over working proof-of-concept code and educated the product team on choice of settings. It was our call setting the difficulty level high and our cavalier attitude towards usability that green-lighted a critical feature absent any empirical evidence of its accuracy, all the while patting ourselves on the back that accessibility requirements were satisfied. Mission accomplished Passport team!

In software engineering we rarely come face-to-face with our errors. Our customers are distant abstractions, caricaturized into helpful stereotypes by marketing: “Abby” is the home-user who prioritizes online safety, “Todd” owns a small-business and appreciates time-saving features while “Fred” the IT administrator is always looking to reduce technology costs. Yet we never get to hear directly from Abby, Fred or Todd on how well our work product actually helps them achieve those objectives. Success can be celebrated in metrics trending up— new registrations, logins per day and less commonly trending down— fewer password resets, less outbound spam originating from Hotmail. Failures are abstract, if not entirely out of sight. Usability studies are the one exception when rank-and-file engineers have an opportunity to meet these mythical “users” in the flesh and recognize beyond doubt when our work products have failed our customers.

CP

On CAPTCHAs and accessibility (part I)

[This is an expanded version of what started out as a Twitter thread]

Fighting spam and failing our customers

When Twitter announced its tweet-by-voice feature, they were probably not expecting the backlash from users with disabilities pointing out that the functionality would be unusable in its present state. It is not the first time technology companies forget about accessibility in the rush to ship features out the door. This is one such story from this blogger’s time at MSFT.

Outbound spam problem

In the early 2000s, MSFT Passport was the identity service for all customer-facing online services the company provided: Hotmail, MSN properties, Xbox Live and even developer-facing services including MSDN. (Later renamed Windows Live and now known simply as MSFT Accounts, not to be confused with a completely unrelated Windows authentication feature named “Passport” that retired in 2016.) This put the Passport team— my team— squarely in the midst of a raging battle against outbound spam. Anyone hosting email has to contend with inbound spam and keeping that constant stream out of their customers inbox. But service providers who give away free email accounts also have to worry about the opposite problem: crooks registering for thousands of such free accounts and enlisting the massive resources available to a large-scale provider like Hotmail to push out their fraudulent messages. Since Passport handled all aspects of identity including account registration, it became the first bulwark against keeping spammers out.

While most problems in economics have to do with pricing, the problem of spam originates with complete absence of cost. If customers had to pay for every piece of email they sent or even charged a monthly subscription fee for the privilege of having a Hotmail account, no spammer would find it profitable to use Hotmail accounts for their campaign. But the Orginal Sin of the web is an unshakeable conviction that every service must be “free,” at least on the surface. To the extent that companies are to make money, this doctrine goes, services shall be monetized indirectly— subsidized by some other profitable line of business such as hardware coupled to the service or increasingly, data-mining customer information for increasingly targeted and intrusive advertising, which begat our present form of Surveillance Capitalism. To the extent charges could be levied, they had to be indirect.

Enter CAPTCHAs. This unwieldy acronym stands for “Completely Automated Public Turing-test to tell Computers and Humans Apart.” In the early 2000s the terminology had not been standardized. At MSFT we used the simpler acronym HIP for Human Interaction Proof. The basic idea is having a puzzle that is easy for humans but difficult for computers to solve. The most common example is recognizing distorted letters inside an image. Solving that puzzle becomes the new toll booth for access to an otherwise “free” service. So there is a price introduced but it is charged in the currency of human cognitive workload. Of course spammers are human too: they can sit down and solve these puzzles all day long— or pay other people in developing countries to do so, as researchers eventually discovered happening in the wild. But they can not scale it the same way any longer: before they could register accounts about as fast as their script could post data to Passport servers. Now each one of those registration attempts must be accompanied by proof that someone somewhere devoted a few seconds worth of attention to solve a puzzle.

Designing CAPTCHAs

So what does an ideal CAPTCHA look like? Recall that the ideal puzzle is easy for humans but difficult for computers. This is a moving target: while our cognitive capacity changes very slowly over generations of evolution, the field of artificial intelligence moves much faster to close the gap.

Screen Shot 2020-06-24 at 10.17.29 AM
A visualization of CAPTCHA design space. Advances in AI continue to push the boundary between problems that are solvable by computers and those that are  only solvable by humans— so far. Ideal CAPTCHAs are just beyond the reach of AI, while still easy for most people to solve.

Philosophically, there is no small measure of irony in computers scientists devising such puzzles. The very idea of a CAPTCHA contradicts common interpretations of the Church-Turing thesis, one of the founding principles of computer science. Named after Alan Turing and his advisor Alanzo Church, the thesis states that the notion of a Turing machine— an idealized theoretical model of computers we can construct today, but with infinite memory— captures the notion of computability.  According to this thesis, any computational problem that is amenable to “solution” by mechanical procedures can be solved by a Turing machine. Its original formulated was squarely in the realm of  mathematics but it did not take long for inevitable connection to physics and philosophy of mind to emerge. One interpretation holds that since the human mind can solve certain complex problems—recognizing faces or understanding language— it ought to be possible to implement the same steps for solving that problem on a computer. In this view popular among AI researchers, there can not be a computational problem that is magically solvable by humans and forever out of reach of algorithms. That makes computer science research on CAPTCHAs somewhat akin to engineers designing perpetual motion machines.

Of course few researchers actually believe such problems fundamentally exist. Instead we are simply exploiting a temporary gap between human and AI capabilities. It is a given that AI will continue to improve and encroach into that space of problems temporarily labelled “only solvable by humans.” In other words, we are operating on borrowed time. It is not surprising that many CAPTCHA designs originated with AI researchers: defense and offense feed each other. Less known is that Microsoft Research was at the forefront of this field in the early 2000s. In addition to designing the Passport CAPTCHA, MSR groups broke CAPTCHAs deployed by Ticketmaster, Yahoo and Google, publishing a paper on the subject. (This blogger reached out to affected companies ahead of time with vulnerability notification and make sure there were no objections to publication.) For CAPTCHAs based on recognizing letters, we knew that simple distortion or playing tricks with colors would not be effective. Neural networks are too good at recognizing stand-alone letters. Instead the key is preventing segmentation: make it difficult for OCR to break up the image into distinct letters, by introducing artificial strokes  that connect the archipelago of letters into one uninterrupted web of pixels.

Gone in 60 seconds: Spambot cracks Live Hotmail CAPTCHA | Ars Technica
Example of original Passport CAPTCHA, from ArsTechnica

Some trial-and-error was necessary to find the right balance. The first version proved way too easy. In fact it turned out that around the same time Microsoft Office introduced an OCR capability for scanning images into a Word document and that feature alone could partially decode some of the CAPTCHAs. Facepalm moment: random feature in one MSFT product 0wns the security feature of another MSFT product. We can only hope this at least spurred the sales— or more likely pirating— of Office 2003 among enterprising spammers. After some tweaks to the difficulty parameters, image CAPTCHAs settled on a healthy middle-ground, stemming the tide of bogus accounts created by spammers without stopping honest customers from signing up.

There was one major problem remaining however: accessibility. Visual CAPTCHAs work fine for users who could see the images. What about customers with visual disabilities?

[continued]

CP

Smart-cards vs USB tokens: esoteric form factors (part III)

[continued from part II]

A smart-card is a smart-card is a…

Once we take away the literal interpretation of “card” out of smart-card, we can see the same underlying IC appearing in a host of other form factors. Here are some examples, roughly in historical order of appearance.

  • Trusted Platform Module or TPM. Defined by the Trusted Computing Group in the early 2000s, TPM provides separate hardware intended to act as root of trust and provide security services such as key management to the primary operating system. Their first major application was Bitlocker full disk-encryption in the ill-fated Windows Vista operating system. In an entertaining example of concepts coming full-circle, Windows 7 introduced the notion of virtual smart-cards which leverage the TPM to simulate smart-cards for scenarios such as remote authentication.
  • Electronic passports, or Machine Readable Travel Documents (MRTD) as they are officially designated by the standardizing body ICAO. This is an unusual scenario where NFC is the only interface to the chip; there is no contact plate to interface with vanilla card readers.
  • Embedded secure elements on mobile devices. While it is possible to view these as “TPM for phones” the integration model tends to be very different. In particular TPMs are tightly integrated into the boot process for PC to provide measured boot functionality, while eSE historically has been limited to a handful of scenarios such as payments. Nokia and other manufacturers experimented with SEs in the late 2000s, before Google Wallet first introduced  it to the US market at scale on Android devices. Apple Pay followed suit a few years later. SE is effectively a dual-interface card. The “contact” interface is permanently connected to the mobile operating system (eg Android or iOS) while the contactless interface is hooked to the NFC antenna, with one caveat: there is an NFC controller in the middle actively involved in the communication. That controller can alter communications, for example directing traffic either to the embedded SE or the host operating system depending on which “card” application is being accessed. By contrast a vanilla card usually has no controller standing between antenna and the chip.

One vulnerability, multiple shapes

An entertaining example of shared heritage of hardware involves the ROCA (Return Of Coppersmith Attack) vulnerability from 2017. Researchers discovered a vulnerability in the RSA key generation logic in Infineon chips. This was not a classic case of randomness failure: the hardware did not lack for entropy for a change. Instead it had a faulty logic that over-constrained the large prime numbers chosen to create the RSA modulus. It turns out that moduli generated as the product of such primes were vulnerable to ancient attack that allowed efficient factoring, breaking the security of such keys. This one bug affected a wide range of products all based on the same underlying platform:

One bug, one secure hardware platform, multiple manifestations.

Beyond form-factor: trusted user interface

There are some USB tokens with additional functionality that at least theoretically improve security compared to using vanilla cards. Most of these involve the introduction of a trusted user interface, augmenting the token with input/output devices so it can communicate information with the user independently of the host. Consider a scenario where digitally signing a message authorized the in transfer of money to a specified bank account. In addition to securing the secret key material that generates those signatures, one would have to be very careful about the messages being signed. An attacker does not need to have access to the key bits to wreak havoc, although that is certainly sufficient. They can trick the authorized owner of the key to sign a message with altered contents, diverting funds into a different account without ever gaining direct access to the private key.

This problem is difficult to solve with standard smart-cards, since the message being signed is delivered by the host system the card is attached to. On-card applications assume the message is correct. With dual-interface cards, there are some kludgy solutions involving the use of two different hosts. For examples the message can be streamed over contact interface initially, but must be confirmed from again over contactless interface. This allows using a second machine to verify that the first one did not submit a bogus message. Note that dual-interface is necessary for this, since the card can not otherwise detect the difference between initial delivery and secondary approval.

A much better solution is to equip the “card” with UI that can display pertinent details about the message being signed along with a mechanism for users to accept or cancel the operation. Feitian LCD PKI token is an example of a USB token with exactly that capability. It features a simple LCD display and buttons. On-board logic parses and extracts crucial fields from messages submitted for signing, such as amount, currency and recipient information in the case of a money transfer. Instead of immediately signing the message, the token displays a summary on the display and waits for affirmative approval from the user via one of the buttons. (Those buttons are physically part of the token itself. Malicious code running on the host can not fake a button press.) Similar ideas have been adopted for cryptocurrency hardware wallets, with one important difference: most of those wallets are not built around a previously validated smart-card platform. The difference is apparent in the sheer number of trivial & embarrassing vulnerabilities in such products that have long been eliminated in more mature market segments such as EMV.

The challenge for all of these designs is that they are necessarily application specific, because the token must make sense of the message and display summarize it in a succinct way for the user to make an informed decision. For all the time standards groups have spent putting angle brackets (XML) or curly braces (JSON) around data, there is no universal standard format for encoding information, much less its semantics. This requires at least some parsing and presentation logic hard-coded in the token itself, in a way that can not be influenced by the host. Otherwise if a malicious host can influence presentation on display, it could also alter the appearance of an unauthorized transaction to appear benign, defeating the whole point of getting having a human in the loop to sanity check what gets signed. Doing this in a general and secure way outside narrow applications such as cryptocurrency is an open problem.

CP

The righteous exploit: Facebook & the ethics of attacking your own customers

Vice magazine recently reported on Facebook using a 0-day exploit for the Tails Linux distribution against one of its own users. The target was under investigation by law enforcement, for a series of highly disturbing criminal acts targeting teenage girls on the platform. Previous efforts to unmask the true identity of the suspect had been unsuccessful because they were accessing Facebook using the Tor anonymizing network. For a change from the usual Facebook narrative involving a platform hopelessly run over by trolls, hate-speech and disinformation, this story has an uplifting conclusion: the exploit works. The suspect is arrested. Yet the story was met with mixed reactions, with many seeming to conflate the ethical issue— was it “right” for Facebook security team to have done this?— with more pragmatic questions around how exactly they went about the objective.

Answering the first question is easy. There is a special place in hell reserved for those who prey on the weak and if these allegations are true, this crook squarely belongs there. There is no reason to doubt that Facebook security team acted in good faith. They had access to ample internal information to judge the credibility of the allegations and conclude that this suspect posed such risk to other people that it warranted going beyond any reasonable definition of “assisting a criminal investigation,” into the uncharted territory of actively attacking their own customer with a 0-day exploit. Ultimately the question of guilt is a matter for courts to decide. Contrary to knee-jerk reactions on social media, Facebook did not act as judge-jury-and-executioner in this episode. The suspect may have been apprehended but due process stands. Mr. Hernandez was still entitled his day in court in front of a real judge with a real jury to argue his innocence, and is unlikely to be strapped into an electric chair by a real executioner anytime soon. (In fact the perp plead guilty earlier this year and currently awaits sentencing.)

More troubling questions about the episode emerge when looking closer at exactly how Facebook collaborated with the FBI in bringing the suspect to justice. These questions are likely to come up again in other contexts, without the benefit of a comparable villain to short-cut the ethical questions. That is, if they have not already come up in other criminal investigations waiting for an enterprising journalists to unearth the story.

Timing

Let’s start with the curious timing of publication: the events chronicled in the article take place from 2015 to 2016. Why did the story come out now? There was no new public disclosure, such as the publication of court documents that could reveal— to our collective surprise— how the FBI magically tracked down the true IP address of the criminal using the Tor network. (A question that incidentally has never been answered satisfactorily in the case of Russ Albricht né Dread Pirate Roberts takedown for Silk Road.) Outline of facts are attributed to current and former Facebook employees speaking as anonymous sources. Why now? A slightly conspiratorial view is that Facebook PR desperately wanted a positive story in the current moment, when the company is under fire for refusing to fact-check disinformation on its platform, a stance made even more difficult to defend after Twitter took an unprecedented step to starting labelling tweets from the President. Facebook may have assumed the story could be a happy distraction and score cheap brownie points: “We will condone rampant political disinformation on our platform, but look over here— we went out of our way to help bust this awful criminal.” It is not uncommon for companies to play journalists this way and intentionally leak the desired narrative at the right time. If that was the calculation, public reaction suggests they badly misjudged the reception.

Facebook & Tor: strange bed-fellows

A second issue that has been over-looked in most accounts of this incident is that for Facebook the challenge of deanonymizing miscreants was an entirely self-inflicted problem. The suspect in question used Tor to access Facebook, leaving no identifiable IP address for law enforcement to pursue. There is a wide-range of opinion on the merits of anonymous access and censorship resistance but there is no question that many companies have decided that more harm than good has originated from anonymizing proxies, whether the vanilla centralized VPN variety or Tor. Netflix has an ongoing arms race to block VPNs while VPN providers compete by advertising that their service grants access to Netflix. Cloudflare has drawn the ire of the privacy community by throwing up time-wasting CAPTCHAs in the way of any user trying to access websites fronted by their CDN. Yet Facebook has been going against the grain by not only allowing Tor access but going much further by making Facebook available as a hidden Tor service and even going so far as to obtain the first SSL certificate ever for a hidden service under “.onion” domain.

Such embrace of Tor is quite puzzling, coming from the poster-child of Surveillance Capitalism with a checkered history of commitment to privacy. Tor gives ordinary citizens the power to access information and services without disclosing private information about themselves, even in the presence of “curious” third-parties trying to scrape together profiles from every available signal. That model makes less sense for accessing a social network that requires identifying yourself and using your real name as the prerequisite to meaningful participation. The implied threat model makes no sense: worrying about hiding your IP address while revealing intimate information about your life on a social network that profits by surveilling its own customers is an incoherent view of privacy. A less charitable view is that Facebook chose to pander to the privacy community in an attempt to white-wash its less than impressive record after multiple miscues, including the Beacon debacle and 2011 FTC settlement.

There is a stronger case to be made around avoiding censorship: direct access to Facebook is frequently blocked by autocratic regimes. Tor is arguably the most reliable way to bypass such restrictions. Granted the assumption that expanding access to Facebook results in a better world all around is a laughably absurd idea today. Between Russian interference in the 2016 election, the Cambridge Analytic scandal, a large-scale data breach, discriminatory advertising, ongoing political disinformation and even ethnic violence being orchestrated via Facebook, one could argue the world just might be better off with fewer people accessing this particular platform. But it is easy to forgive Facebook for this bit of self-serving naïveté in 2014, a time when technology companies were still lionized, their negative externalities yet to manifest themselves.

How much of Facebook usage over Tor is legitimate and how much is criminal behavior— such as Hernandez case— disinformation, fraud and spam? As with most facts about Facebook, these data points are not known outside the company. (It is possible they are not even known inside Facebook. For all their unparalleled data-mining capabilities, technology companies have a knack for not posing questions that may have inconvenient answers— such as what percent of accounts are fake, what fraction of advertising clicks are engineered by bots and how much activity from your vaunted Tor hidden-service is malicious.) What is undisputed is that the crook repeatedly registered new accounts after being booted off the platform. Without having a way to identify the miscreant when he returned, Facebook was playing a game of whack-a-mole with these accounts. Services have many options between outright blocking anonymizing proxies and giving them unfettered access to the platform. For example, users could be subject to additional checks or their access to high-risk features— such as unsolicited messaging of other users or video sharing, both implicated in this incident— can be restricted until the account builds sufficient reputation over time or existing accounts vouch for it.

Crossing the Rubicon

Putting aside the question of whether Facebook could have prevented these actions ahead of time, we turn to the more fraught issue of response. Reading between the lines of the Vice article, victims referred the matter to law enforcement and the FBI initiated a criminal  investigation. In these scenarios it is common for the company in question to be subpoenaed for “all relevant information” related to the suspect, in order to identify them in real life. This is where the use Tor frustrated the investigation. IP addresses are one of the most reliable pieces of forensic evidence that can be collected about actions occurring online. In most cases the IP address used by the person of interest directly leads to their residence or office. In other cases it may lead to a shared network such as a public library or coffee shop, in which case a little more sleuthing is necessary, perhaps looking at nearby video from surveillance cameras, license plate readers or any payments made at that establishment using credit cards. With Tor, the trail stops cold at the Tor exit node. If the user had instead used a commercial VPN service, there is a fighting chance the operator of the service can be subpoenaed for records. With a decentralized system such as Tor, there are too many possible nodes, distributed all over the world in different jurisdictions with no single party that could be held accountable. In fact, that is exactly the strength of Tor and why it is so valuable when used in defense of free-speech and privacy.

Facebook security team could have stopped there after handing over what little information they had to the FBI. Instead they decided to go further and actively work on unmasking the identity of the customer. This is a difficult stance. In the opinion of this blogger, it is ethically the correct one. The miscreant in question caused significant harm to young, vulnerable individuals. This harm would have continued as long as the perp was allowed to operate on the platform. Absent the appetite to walk-back on the seemingly inviolable commitment to making Facebook available over Tor, the company had no choice other than going on the offensive with an exploit.

Sourcing the exploit

Once the decision is made to pursue active attacks, the only question becomes how. There is a wide range of options. On the very low-tech side of the spectrum, Facebook employees could impersonate a victim in chat messages and try to social-engineer identifying information out of the suspect. There is no mention in the Vice article of such tactics being attempted. It is unlikely that a perp with meticulous attention to opsec would reveal identifying information in a moment of carelessness. What is implied by the article is that FBI immediately reached for high-tech solutions in their arsenal. The first exploit attempt failed, likely because the it was designed for a different platform— operating system and browser combination— than the esoteric setup this crook had involving the Tails Linux distribution.

Luckily there is no shortage of vulnerabilities to exploit in software. Take #2 witnessed Facebook contracting an “outside vendor” to develop a custom exploit chain for the specific platform used by the suspect. This is a questionable move, because going outside to source a brand-new exploit all but guarantees the independent availability of that exploit for others. Sure, Facebook can contractually demand “exclusivity” as a condition for commissioning the work, but let’s not kid ourselves. In the market for exploits, there is no honor among thieves. It is unclear if this outsourcing was a deliberate decision to distance the Facebook security team itself from exploit development or they simply lack the talent in house. (If this were Google, one expects Project Zero cranking out a reliable exploit in an afternoon’s work.)

Pulling the trigger

The next questionable step involved the actual delivery of the exploit, although Facebook may not have had any choice in the matter. According to Vice, Facebook handed over the exploit to the FBI for eventual delivery to the perp. It is as if a locksmith asked to open the door to one particular household to conduct a lawful search, simply hands over the master-key to the policemen and goes home. At this point Facebook has gone far beyond the original mission of unmasking of one noxious criminal: they handed over a 0-day exploit to the FBI, ready for use in any other situation the agency deems appropriate. (Senator Wyden is quoted in the Vice article questioning exactly this logic, asking whether the FBI later submitted the exploit the Vulnerabilities Equity Process.)

Legally the company may have had no other option. In an ideal world, Facebook holds on to the exploit— they paid for it, after all— and delivers it to the suspect directly, under tacit agreement of the FBI, with some form of immunity against prosecution for what would otherwise be a criminal act committed in the process: Facebook breaking into a machine it is not authorized to access. It is unlikely that option exists in the real world or even if it did, that the FBI would willingly pass on the opportunity to add a shiny new 0-day to its arsenal at no cost.

Disarming the exploit?

Given that Facebook had already sourced the exploit from a third-party, there is no guarantee the FBI would not have received a copy through alternate channels, even if Facebook managed to hold on to it internally. That brings up the most problematic part of this episode: vulnerability disclosure. According to Vice, Facebook security team decided against formal vulnerability notification to Tails was required because “the vulnerable code in question had already been removed in the next version.”

That is either a weak after-the-fact excuse for inaction or a stunning lapse of judgment. There is a material difference between a routine software update that inadvertently fixes a critical security vulnerability (or worse fixes it silently, deliberately trying to hide its existence) and one that is actually billed as a critical security update. In one case, users are put on notice that there is an urgency to applying the update in order to protect their systems.

Given that the exploit was already in the hands of the FBI and likely being resold by the original author, this was the only option available to Facebook to neutralize its downstream effects. Had they disclosed the issue to the Tails team after the crook was apprehended, it would have been a great example of responsible use of an exploit to achieve a limited objective with every ethical justification: delivering a criminal suspect into the hands of the justice system. Instead Facebook gave away a free exploit to the FBI knowing full well it can be used in completely unrelated investigations over which the company has no say. If it is used to bring down another Hernandez or comparable offender, society is better off and we can all cheer from the sidelines for another judicious use of an exploit. If the next target is an immigrant union organizer wanted for jay-walking or a Black Lives Matter activist singled out for surveillance based on her race, the same argument can not be made. From the moment this exploit was brought into existence until every last vulnerable Tails instance has been patched, Facebook security team bears some responsibility in the outcomes, good or bad.

It turns out trafficking in exploits is not that different from connecting the world’s population and giving everyone a platform to spread their ideas— without first stopping to ask whether they going to use that capability for charity or malice.

CP

Smart-cards vs USB tokens: optimizing for logical access (part II)

[continued from part I]

The problem with cards

Card form-factor or “ID1” as it is officially designated, is great for converged access: using the same credential for physical and logical access. Cards can open real doors in physical world life and virtual doors online. But they have an ungainly requirement in  smart-card readers to function properly. For physical access this is barely noticed: there is a badge-reader mounted somewhere near the door that everyone uses. Employees only need to remember to bring their card, not their own reader. For logical access, it is more problematic. While card-readers can be quite compact these days, it is still one more peripheral to carry around. Every person who needs to access an online resource gated by the card— connect to the company VPN, login to a protected website or decrypt a piece of encrypted email— also needs a reader. For the disappearing breed of fixed-base equipment such as desktop PCs and workstation, one could simply have a reader permanently glued to every desk.

Yet the modern work-force is highly mobile. Many companies only issue laptops to their employees or at least expect that their personnel can function just equally effectively when they are away from the office— good luck getting to that workstation in the office during an extended pandemic lockdown. While a handful of laptops can be configured with built-in readers, the vast majority are going to require a reader dangerously dangling from the side, ready to get snapped off, hanging on to a USB-to-USB-C adapter since most readers were not designed for the newer ports. There is also the hardware compatibility issue to worry about since most manufacturers used to target Windows and rely on automatic driver installation through plug & play. (This is largely a solved problem today. Most readers comply with the CCID standard which has solid support through the open-source libccid package on Linux & OSX. Even special snowflake hardware is accompanied by drivers for Linux.)

USB tokens

This is where USB tokens come in handy. Tokens combine the reader and “card” into a single device. That might seem more complex and fragile but it actually simplifies detection from the point of view of the operating system. A reader may or may not have a card present, so the operating system must handle insertion and removal events. USB tokens appear as a reader with a card permanently present, removing one variable from the equation. In addition to featuring the same type of secure IC from a card, these devices also have a USB controller sitting in front of that chip to mediate communication. Usually the controller does nothing more fancy than “protocol arbitrage:” taking messages delivered over USB from the host and relaying them to the secure IC using ISO7816 which is the common protocol supported by smart-cards. But sometimes controllers can augment functionality, by presenting new interfaces such as simulating a keyboard to inject one-time passcodes.

Some examples such as Safenet eToken used in SWIFT authentication define their own card application standard. More often vendors choose to follow an existing standard that enjoys widespread software support. The US government PIV standard is a natural choice, given its ubiquity and out-of-box support on Windows without requiring any additional driver installation. In late 2000s GoldKey became an early example of offering a PIV implementation in USB token format; they also went to the trouble of getting FIPS140 certification for their hardware. Taglio PIVKey followed shortly afterwards with with USB tokens based on Feitian and later NXP platforms. Eventually other vendors such as Yubico copied the same idea by adding PIV functionality to their existing line.

In principle most USB tokens have no intrinsic functionality or security advantages over the equivalent hardware in card form. They are just a repackaging of the exact same secure IC running the exact same application. The difficulty of extracting secrets from the token can not be appreciably more difficult than extracting secrets from the equivalent card. If anything, the introduction of an additional controller to handle USB communication can only make matters worse. That controller is privy to sensitive information flowing across the interface. For example when the user enters a PIN to unlock the card, that information is visible to the USB controller. Likewise if the token is used for encryption, all decrypted messages pass through the secondary controller. So the USB token arguably has a larger attack surface involving an additional chip to target. Unlike the smart-card IC which has been developed with security and tamper resistance objectives, this second chip is a sitting duck.

Usability matters

Yet from a deployment perspective, USB tokens greatly simplify rolling out strong two-factor authentication in an enterprise. By eliminating the ungainly card reader, they improve usability. Employees only need to carry around one piece of hardware that is easily transported and could even remain permanently attached to their laptop, albeit at the risk of slightly increasing certain risks from compromised devices. Examples:

  • In 2014 Airbnb began issuing GoldKey PIV tokens to employees for production SSH access. Airbnb fleet is almost exclusively based on Macbooks. While the office space included fixed infrastructure such as monitors and docking stations at every desk, none of that would have been available for employees working from home or traveling— not uncommon for a company in the hospitality business.
  • Regulated cryptocurrency exchange & custodian Gemini issues PIV tokens to employees for access to restricted systems used for administering the exchange

The common denominator for both of these scenarios is that logical access takes precedence over physical access. Recall that CAC & PIV programs came to life under auspices of the US defense department. Defense use-cases are traditionally driven by a territorial mindset of controlling physical space and limiting infiltration of that area by flesh-and-blood adversaries. That makes sense when valuable assets are tangible objects, such as weapons, ammunition and power plants. Technology and finance companies have a different threat model: their most valuable assets are not material in nature. It is not the office building or literal bars of gold sitting in a vault that need to be defended; even companies directly trading in commodities such as gold and oil frequently outsource actual custody of raw material to third-parties. Instead the nightmare scenarios revolve around remote attackers getting access to digital assets: accessing customer information, stealing intellectual property or manipulating payment systems to inflict direct monetary damages. When locking down logical access is the overarching goal,  usability and deployment advantages of tokens outweigh their incompatibility with physical-access infrastructure.

[continued]

CP

Updated: June 20th, with examples of hardware token models