Making Better Episode 20
(music) Welcome to the Making Better Podcast, interviewing some of the world’s finest thinkers about a more optimistic future. Now, here are your hosts, Chris Hofstader and Dr. Francis DiDonato.
Chris: Hi, I’m Chris Hofstader
Francis: Hey, I’m Francis DiDonato
Chris2: ..and I’m Chris Smart, the producer of Making Better. Due to a technical glitch, we lost our intro that we had recorded for this episode, so you’re going to have to listen to me a few seconds. Our guest this time is Andreas Steffik, professor and inventor of the Quorum programming language, which is 100% accessible to blind or visually impaired programmers. Quorum is also a very high-level language, excellent for teaching kids how to code, and Quorum is the first programming language developed using the scientific method. And without further ado, here is Episode 20.
Chris: Andrea Steffik, Welcome to Making Better podcast!
Andrea: Thanks for having me, guys!
Chris: You’re most well known for having invented a programming language called Quorum. Now, a lot of our listeners might not even know what a programming language is—I mean, they know how to use their iPhone, but they don’t know how it works—so if we could maybe even start with the absolute basic of what’s the difference between a programming language and a spoken or written language.
Andrea: Yeah, sure. So when people write computer software, oftentimes computers are really picky, right? They don’t necessarily understand natural language in the way that you might see on Star Trek or on science fiction shows, so oftentimes computer scientists have derived ways to tell the computer, effectively, what they want it to do. But here’s the catch: even though we knew pretty quickly after people like Alan Turing were inventing a lot of ideas mathematically about programming, even though we had some mathematical understanding of how to tell a computer what to do, we had almost no understanding of how to tell humans how to tell computers what to do. And so, for example, in a programming language like C++, if you want to tell the computer to do something ten times, literally just that. You might say a phrase like, “for (anti=0; i<10; i++){}” That’s a lot of weird verbiage and symbols to represent the concept of just doing something over and over again; but in fact, that’s kind of how it derives. Now the idea in a programming language is, it’s not just the human interaction, it’s actually also what happens when it gets sent down to, like, a chip. You take those sort of like weird words and symbols and stuff like that, and then it gets translated down to ones and zeros, and those ones and zeros are the thing that actually tell your iPhone or your Windows machine or Linux or Mac or whatever you have, what to do when you use Siri, or when you load up and app or a game or anything like that. So basically, it’s the translation between what a human wants it to do, and then what a machine wants it to do, and the whole tricky part about Quorum is we’re trying to gather evidence to figure out how to make it easier for those pesky humans that are in the process.
Chris: My life as a programmer started with, actually my very first language was PDPA Assembly language, and I’d go on to learn—you know, I did a lot of programming in C and probably about 25 different Assembly languages, I was a low-level, silicon-under-the-fingernails kind of hacker—and those languages are incredibly arcane. So, you used the scientific method to design your language, Quorum. Could you tell us something about that process? Because the languages I’m accustomed to were pretty much designed by electrical engineers.
Andrea: Yeah, that’s true. So, let me give you a little bit of context. You might assume, given that there’s a lot of [sobejrs?] that are paid in the United States, that this issue of how to make it easier for people would be really well studied. Let me sort of formalize this for the United States, at least. So if you take Bureau of Labor Statistics data and then you look at how many people are basically in the ballpark of computer scientists, like software engineers and there’s a couple other categories, it’s over 300 billion dollars a year that we pay software engineers to develop software. So, it’s a lot of money. Now here’s the thing—what that means is, is that if there’s anything wrong with programming languages for any one of them, that is causing productivity issues, confusion, lost hours—that means that, at scale, it can be very expensive to try and write software sort of like with one hand behind your back, if you will, if the languages are slowing you down. So here’s the thing: you might then think that because that’s so expensive, that companies would be interested, that academics would be interested. But in fact, there’s this wonderful study by this guy call Antihanikayanumho, who’s out of Finland, and a number of us in the academic literature were basically saying, hey, we can’t find the evidence. Where’s the data? You guys are all scientists, why don’t you have data on this issue? It’s a lot of money. And Anti** actually didn’t believe us, so in Finland he basically sat around, I swear, for about four years, paper by paper, reading them in the academic literature, like Programming Language communities, Software Engineering places like that, paper by paper, counting, is there evidence in this peer-reviewed, top conference paper, or is there no evidence in this paper? And so then he did that for the years between, like, 1950 and up until modern times, in his case he stopped in 2012 because that’s the nature of his finishing a PhD. And it turns out, for this more than $300 billion in wages that we spend, it turns out there’s only 22 experiments between 1950 and 2012. Now to put this in perspective, in fields like medicine, 22 would be adorable, right? So like between 1990 and 2001, there were something like 110,485 randomized controlled trials. The point is, we have this huge, huge problem related to wages that any tiny little increment of productivity lost is actually very expensive at scale, because engineers are expensive to hire. However, we have almost no evidence on what the impact is on people, so we don’t even know how much of this wage data is actually being wasted because we don’t have any data. So part of the Quorum project was first realizing that, and then the next part is figuring out how on earth do you actually study something as complex as how people invent things to try to lower that curve. And that’s a whole other topic that I imagine we’ll get into. That’s sort of the 10,000 foot view of the evidence and why it matters and stuff like that, and then the details are tricky, too, in a different way.
Chris: Well, how did you use the Scientific Method to design Quorum?
Andrea: Right. So when we first designed it, even though Antihani’s work hadn’t come out yet, we realized that there wasn’t much data. And the truth is, we didn’t really know what to do, but we were doing lots of observations of people—like, we conducted a lot of studies with blind children, and blind children had a special difficulty because in audio, these languages are often even more esoteric—you know, you have to state all the semicolons, screen-readers skip characters, there’s all sorts of complicated stuff. But in addition, because there was no studies, it really left us to ask, well how do we even run such studies? And of course, in computer science, if you go get a PhD in the field, you’re not trained in how to conduct empirical studies, that’s very rare. So what I did was, I thought well I better get help. So I spent several years just sitting around psychology departments trying to figure out what they do, to figure out how they investigate complex problems. And then eventually, although it took a few years to figure out the details, we started making scientific experiments that are related to the medical models. And so, this means especially a couple techniques: one is, we conduct surveys for things that we can’t easily manipulate—when we’e just trying to get subjective data. So this is useful for things like what keyword should I use to represent this idea, and I’ll give you a quick example of that. In C++, I mentioned that if you want to have something do something over and over again, you say “for (anti=0; i<10; i++)” and that means to do something ten times. But on quorum, after doing extensive surveys and then replicating them and doing all sorts of stuff, we ended up with much simpler phrases for many of these constructs. So in Quorum, the exact same code that does the same thing, is just as efficient, is: “repeat 10 times” right? Which is much simpler. And you literally just write then in the computer program, “repeat 10 times”, and it does the same thing as the previous code. Now then what we’ve done is we do that with every part of the language, or as many as we have time to conduct studies on, and then once the surveys are done, we then move to features using a procedure called randomized controlled trials, which is a really complicated topic, but basically involves pitting alternative features against each other and then whatever wins, goes in the language and whatever loses gets removed from the language over time in a deprecation process.
Chris: Part of what you did designing Quorum developed a control language called Rando, I think.
Andrea: Oh god, this is true.
Chris: What was it called?
Andrea: Random-o. Which—was actually not my idea, it was a colleague of mine named Bill White, who unfortunately passed away in the last year. But I wanted to call the thing “Ridicu-lo” but my friend told me you better watch it, in peer review they’re going to yell at you. He was like, well, it’s all random, why don’t you just call it “Random-o”? So.
Chris: And what did Random-o look like?
Andrea: So basically what we did, is we had conducted all these surveys of what words to use. But then, I had this idea that from the medical literature, ‘cause I was starting to read, that maybe what we want to do is we want to compare languages with a baseline, right? Like say, if I’m testing Python or Assembly or any other of these programming languages, I want sort of like a baseline to compare them all to, to figure out what characters and symbols might sense compared to a baseline. But the question is, what the heck is a baseline? So when you look at the history of medicine, that often became placebos, and placebos are a complicated topic, but often they involve taking like a sugar pill or something like that. But there’s no programming language “sugar pill,” that doesn’t compute, right? So that meant there was a couple of options: one is, there are programming languages that have been designed to be intentionally difficult for people—one of them is called “whitespace” and there’s another one called “brainfuck.” And these particular languages could be used as a baseline, but the problem is, they’re sort of designed to be confusing, and it felt kind of unfair. It wouldn’t be a fair comparison to have a baseline—it’s sort of like, instead of giving someone a sugar pill, it’s like poisoning them. Yeah, you’re going to do better than murdering your patients. So what we thought to do instead was, once we had designed Quorum, we then ripped out all the symbols, and with somebody in my lab—Suzanna Sebert was her name, at the time—she basically sat around effectively rolling electronic dice and she randomly chose symbols to represent the syntax of the programming language. For example, instead of “repeat,” it might be “\” it might be “#” it might be something else. And so, the language, when you look at it, of course, is absolutely absurd, literally randomly chosen. And so you might think that a language that is totally designed randomly would do very badly in studies, and that is of course true; however, it turns out some programming languages don’t do better than a random..
Chris: I think I saw you at a C-SUN presentation where you said the Random-o outperformed Pearl, which I can understand entirely, because Pearl is completely incomprehensible.
Andrea: Yeah, right. Well yeah. So in the first study that we ran, which was really small, we only had like eighteen people in that study, it was a tiny, tiny pilot study. And we had this hint in the data that Pearl might not be much better than random. But the problem is, people in my lab, we sort of thought, how could that be true? Even though Pearl I think has a reputation for being incomprehensible, it was intentionally designed by a human being to look like that. And so we were always really hesitant to just say they did a bad job, because that’s a subjective thing, but the reputation for it being hard to understand could be well-deserved, but we didn’t believe it. So we ran a replication study that was much larger, and compared, I think it was six languages—in this case I think it was Pearl, Python, Java, Quorum, Random-o and maybe one other, but I forget. In any case, the study that we ran the second time, the replication, with the Pearl result was effectively identical, on a new sample of people. We also found that Java actually did slightly worse than Pearl. To a lot of people, that sounds a little surprising, there’s a couple reasons. One, Java is actually required nationally for students in high school as part of the computer science A-standard through the College Board. So, the College Board literally chose a language that studies show does no better for human beings in terms of comprehension, than one where my lab sat around rolling dice and chose the symbols randomly—which is kind of terrible. But in addition, we sort of figured out why from that study, why it happens, or at least we suspect we have a pretty good hypothesis as to why. And the reason is, a lot of the individual symbols that were chosen in programming languages are actually common across many of them; so Pearl and Python are different, but they actually have many commonalities, like they both use the word “for” for looping constructs. They both have certain kinds of brace structures, I mean Python doesn’t but Java does. And it turns out a lot of the decisions that were made across the board related to C-style syntax, which is where a lot of these come from, actually made bad initial decisions and then a lot of programming languages effectively just copied them.
Chris: Kernigan and Ritchie just based the C programming language on mathematical logic—I mean, if you’ve taken a mathematical logic course, you kind of know C automatically.
Andrea: Yeah. So, very briefly, I was at a DARPA meeting and Brian Kernigan was there, and somebody asked him how he derived the symbols—and I swear to you, the study that we had written wasn’t out yet at the time, but I’d just finished it and we hadn’t submitted it for publication–“but Brian, but how did you figure out how to choose all the symbols and stuff?” He’s like, “well look, we didn’t know that C was going to become a standard, so we just kind of chose randomly.” And I swear to you, he said those actual words, but of course it was a DARPA meeting so, you know, it’s not like recorded or anything like that.
Chris: Now you’ve gotten to the point where you’ve developed Quorum. What is it, other than efficiency—is it easier to learn, is it…can I do more with fewer lines of code, what is it that makes Quorum special?
Andrea: The short answer is that, as the language progresses over time, it gets easier to use as we gather more data and make changes. So, I’ll give you an example of a change we’re making right now, because it will give you a sense of the process we do to make things easier. There’s this wonderful paper by Neil Brown in the UK, talking about the errors that programming languages give out. And the errors that languages give out are often really esoteric, they have like weird symbols like “__cdeco::55123” or something like that. I’m making up those exact symbols, but the idea is they’re often very esoteric, and they don’t give you common sense English explanations, like hey you missed a semi-colon, or this character looks wrong. They’re not really like that. And interestingly, there’s some evidence from a different scholar in Ireland, named Brett Becker, and his data shows pretty clear evidence—outside of our research group—that these errors can be made a lot simpler, and more crucially, that if do that it makes it easier to learn programming languages in the classroom. So if my team, we read Brett’s work and started talking to him to figure out what we do to make things easier. And it turns out that, when we investigate the errors that people get in Quorum—which we know because people use it online all the time, and so we can see the kinds of errors people have—it turns out that 70% of the errors that are in Quorum right now, today, that we need to fix this, are actually generated by a tool behind the scenes called Antler, this sort of parsing tool Terrance Parr in San Francisco developed. But it turns out that the specific regions of Antler that trigger these errors give pretty esoteric things, and it’s pretty obvious, looking at the data, that these are going to be difficult for people to use because of Brett’s data and because of Neil’s data. So basically, what I have a graduate student doing right now is doing all sorts of behind the scenes calculations about what triggers these errors, under what conditions and stuff like that for people, so then we’ll go back and then review them, do edit passes, go through peer review related to what these errors are, with the goal of, within a release or two, making it so people can solve these errors a little bit faster, which we’ll measure in our lab. So that’s an example sort of like an upcoming study, but a lot of Quorum is built with the same philosophy. Like, somebody will email us and say, hey, I wish you had this feature, and we’ll test it in the lab, and it will either work or it won’t, in terms of being easier. I’ll give you one brief example of that one. There’s a topic in programming languages called lamdas—a lamda is a totally esoteric, totally sort of in the business kind of way to have a function that doesn’t have a name. And it sounds like some fancy word for it, but I swear to you that’s really all it is. It’s a function, that doesn’t really have a name, and it computer science you can kind of pass these around in certain ways, they can be transferred and there’s all kinds of funny things you can do with this. Now interestingly, when the programming language C++ added them to the language, they wrote this long white paper about why they were important, and why they were adjusting the language and the purpose and all this kind of stuff. And so we were curious, like well maybe we should have lamdas in Quorum, and some of the people in my lab were arguing pretty vociferously for them because they liked them. But people’s likes don’t always match people’s productivity or things like that. So we conducted a test, and we compared exactly to the criterion that the people in the white paper on C++ claimed would be benefits, and then we tested it with a whole bunch of different kinds of people. Notably, we tackled freshmen, sophomores, juniors and seniors at a university, because we’re testing in college range in that case. But in addition, we also recruited from a group of people that are sort of like really aficionados of lamdas—like they eat and breathe these things, they’re part of a group called the lamda lounge, they get together and talk about lamdas—they like lamdas. And so you would think that a group like this would have significant bias, and as such it’s interesting to test with them, just to kind of see what happens. But when you really pull out the data, and you conduct the experiment, as it turns out, lamdas really don’t help, at least in the context of the study that we did. And therefore, when we went tot he next design phase of Quorum, lamdas didn’t get included, because the study showed that they really didn’t help that much. Does that make sense? Sometimes computer programming is really like really esoteric, so it’s sometimes hard to figure out how to translate into English, but I’m doing my best.
Francis: Back in the day, I guess it was 2016, when we had a President that actually could think, you were honored at the White House as a White House Champion of Change.
Andrea: Yes sir.
Francis: And that’s really impressive. So I’m wondering, what is it about your program that caught the White House’s eye and why would this program be honored (and you) as a Champion of Change?
Andrea: You never really know for sure, but I’ll tell you the story and then tell you what they told me—what they thought—and then you can kind of decide yourself. So, I was literally sitting on Dumbo with my daughter at Disney World, or something like that. And I got off, my phone was beeping and I was annoyed because I shouldn’t have even had my phone with me. But I got this email from the White House, and I was like, that doesn’t happen. I thought it was fake, so I effectively mailed them back (because it’s hard to spoof a mailback from an address that’s a dot-gov), and they’re like, no, this is real, blah-blah-blah, you should do this thing. And I’ve been nominated for this award by someone, and I didn’t know who, and I couldn’t figure out who, etc. etc. And that ends up being a long story that I won’t get into, but in any case, by that January, I’d found out that I’d won this thing, and I still didn’t really in my head have a good sense of why, exactly what it was for, why or stuff like that, but the White House basically started talking with us about the details of why individual people on the award and I didn’t know fully until I got there. But when started developing Quorum, there was really two things, I think, and that they expressed to me (the staffers and such) that caught their eye. Number one is, when we first started Quorum, there was almost no initiatives in the United States to include people with disabilities in programming. And this is a big deal, because people with disabilities make up, depending on the number you want to cite, between 13 and 15 percent of people broadly. So like, it’s a lot of folks, you don’t just want to not include them. But in addition, in K through 12, it’s actually required by law that any kid with a disability, no matter who you are, whether it’s blindness, deafness, doesn’t matter—you should be able to participate in whatever academic stuff is going on. So if that’s programming, that’s important, if it’s history, you shouldn’t be excluded, that seems unfair. When we first started Quorum, the very first year we were testing with like, people with disabilities, because I just thought that was really interesting, they had cool problems, it was really fun going and talking to a group of blind children and just learning what they care about, what they’re interested in and—it was just a lot of fun, and it ended up sparking this sort of like movement where now there’s schools all over the country, at schools for the blind, that are teaching programming. Now Quorum is used by way, way more people than just at schools for the blind. I care very deeply about that, and so everything we do we try to make just hyper-accessible, as much as we can. Nothing’s perfect, but we try. That’s number one, the second one was, when I talked to people and the staffers, they said, you know, this problem where there’s 300 or more billion dollars in wages and we aren’t gathering much evidence, that seems like a lot of money, that seems significant. So they had me write up a thing talking both about groups of disabilities and causing change in a K-12 community to get more people involved, but in addition change in an academic community to try to push other scholars to be gathering more and better evidence over time. I think it was those two things that caught their eyes, but you know, you never know for sure. It was a little humbling, to say the least. I was pretty surprised to get that award.
Francis: Well nowadays you have to be a flat earther or something to get recognized for your scientific achievements at the White House.
Andrea: No kidding. Have you seen that flat earth movie on Netflix? It’s just…
Francis: Not yet.
Andrea: I forget what it’s called, but it is really crazy how conspiracy theories have become sort of the norm—not just at the White House, although that’s true. But I think also like how algorithms on YouTube and other places sort of highlight these sort of crazy, crazy ideas, and it’s hard to tamp down. Climate change denialism is very well funded—that’s insane, but it is what it is. It’s weird that, like, we balance freedom of speech so much that we’re allowed to shout conspiracy theories from the rooftops—which sounds OK, but on the other hand, it’s kind of dangerous, because if those people control the White House, they also change policy which impacts all of us.
Chris: Getting back a bit to Quorum, how is it being used in the wild now? I mean, it’s been around for a few years, so people must be doing something with it.
Andrea: Yeah, quite a bit. So the strategy in the lab is sort of, it’s kind of a dumb name, but it is what it is—we call it push up, push down, and when we started we were targeting only—and no one else—a small number of schools for the blind. And that involved, in the very first year, about four or five kids, somewhere around there, so like, nobody, right? And that made sense, because Quorum was brand new, it barely worked—I mean, we had just started the thing—and that was sort of where I began. But, interestingly, it started to grow at first over the next couple of years in the blind community, and the number of users ballpark-doubled pretty much every year, for year-on-year, for many years. Nowadays, if you include only people online, I think it’s something like 81,000 users per year, is what it was for this last year. Now the question is, what do all these people do with it? And the answer is, I’m not totally sure, but we think it’s mostly schools, and that’s on purpose because it’s an evidence-based programming language, we want to be able to change it, and it’s really convenient to be able to change it in relation to the school cycle over the summer. So for example, we gather evidence over the school year, we do whatever we’re going to do, and then over the summer we’ll sort of flip a switch, change over all the curriculum, change over the language, and then people adapt during our trainings and stuff like that over the year for the new change. That’s so complicated, but generally speaking that’s gone really well, and it’s allowed us to sort of find a nice balance between being used in practice, and also maintaining an evidence base that can allow us to make adjustments slowly over time. So what is it used for? I think mostly K-12 computer science would be the most accurate answer. However, we have a lot of technologies that are useful even at the professional level, we’re just not targeting those groups yet because we kind of want to wait until a few more years, as the language develops, so that we can basically make adjustments over time. In other words, the closer it gets to professional practice and use by companies, they’re not going to want it to change very quickly, and so we kind of are trying to find that balance with the language. Let the user base grow, but try to grow it on the younger side first. That’s our push down, and then later push up.
Chris: And what specifically did you do to make it accessible to people with disabilities?
Andrea: When we started talking to blind children, I spent a lot of time just sitting there asking them, like “hey, you’re 14 you’ve never used a computer before, and like you don’t know much about screen readers either, but now you’re a programmer, and what do you want to do with it?” And so, totally unfair, unreasonable questions, but we were just trying to talk to kids and see—you know, if you’re a kid, you’re blind, and you want to do stuff, what kind of features do you want, what would help you? So that led us to a couple directions that we’re either still working on today, or that we do—and then also there’s a few papers that have come out in the last ten years that have shown pretty strong evidence about what features are helpful to people when they code. And so, I’ll give you a couple examples. First, I’ll do the code ones, because they’re pretty easy to understand. And if your listeners that aren’t blind, that haven’t used stuff, when you use a text editor, it turns out that like the way that you handle keyboard support is really important for people that are blind. There’s all sorts of special keys that let you navigate in certain ways, that let you get context information, and stuff like that. In programming, those special keys don’t really have the same meaning, and so—there’s a wonderful study by a woman named Katie Baker that showed pretty good evidence about how you could redesign that navigation so that it would make it easier for people to get around in a programming interface. And then that was followed up by this wonderful disertation by the guy named Amir Armali, who also works at Google but is, like, a rockstar, just absolutely fantastic person and academic. And he basically extended Katie’s stuff to show that you should modify these keys in another fancy way. And so one of the things that we do is, as other academics work in this space, we steal their ideas as much as possible to make sure that we’re sort of conforming to the evidence about what helps people with disabilities most, not just a blindness specific feature. But the short answer with that navigation is in our tools, one of which is called Quorum Studio, there’s a feature called “smart navigation,” which is effectively just Amir Armali’s dissertation work, but coordinated inside of that environment—it lets you bounce between functions in a certain way, and bounce between parts of the code in a very known, specified manner. So that’s one. That’s just navigation, there’s evidence it helps, but that’s just one feature. One of the other things that, when we talked to blind children, that they wanted to do is that kids consistently and repeatedly wanted to be able to design computer games. And when we asked them, we said, oh yeah, you want to make audio games? Because maybe a child is fully blind—and almost universally, these kids were like, no, I don’t want to make audio games, I want to make games that I can play with my friend, and my friend happens to be able to see. And so, that’s just a claim I heard over and over again when I talked to kids in the wild. And the problem is, I didn’t know how to make computer graphics accessible, not just to people that are blind but there’s a lot of complexity in making games accessible in general, for lots of people with different kinds of disabilities. But eventually we figured it out, and so we did some collaborative work with our lab and the University of Washington, especially this guy named Richard Labner, and we also eventually did some collaboration with Microsoft on some of their back end, so this stuff is pretty complex technologies. And we derived a methodology which you can do any kinds—and I mean any—kind of complex 2D or 3D graphics, fully blind-accessibly. And the idea is actually really simple; basically we have a graphics layer that can render or basically put on the screen any kind of 3D graphic—so this can be like a cube, this could be like a spinning, fire-breathing dragon, this could be and architectural plan, this could be engineering or CAD diagrams or anything like that. And inside of our applications, because of this sort of pseudo magical technology we have behind the scenes that took me like five years to figure out, when the screen reader gets it, it has no idea that there’s this complex sort of 2D and 3D graphic stuff going on behind the scenes, but it knows absolutely what those things are and allows screen reader users to interact with it at the sort of native operating system level, in the general sense, using any screen reader, at least on Windows, for right now. So, the application, at a laymen’s level, is basically you can make 3D games and have 3D game level editors and stuff like that, and you can do it even if you can’t se the screen.
Chris: Francis’ son has a great interest in playing video games, maybe he’d be interested in learning Quorum and making his own?
Andrea: Love it! That’d be great. I’d be happy to share anything. The 3D graphics features comes out in Quorum Studio 2.0, which will be in July* but we have—the beta is working now and we’ve been starting to do some webcasts and stuff on it. So I’d be happy to share those with you or your son, and we can put him in the beta, too, If he just wants to give it a shot.
Francis: Sounds great. He’s a real science nerd, and I think he would really enjoy coming up with a game that doesn’t involve killing people.
Andrea: Ha ha. Sorry, we only allow games to be created that kill people, that’s the rule, I’m sorry…
Francis: (chuckles) It’s a stretch, I know.. What the original inspiration for doing it, and was it intimidating to think that you could start your own language and people would avail themselves of it? Like, how do you go from having a language to having people know about it and use it, I guess is the second part of my question.
Andrea: And so the first part was, how did it get started, and the second was…
Francis: Yeah, what was the inspiration, originally? Because to undertake something that big, I imagine you had to have one of those moments where you were just really inspired.
Andrea: When I was an undergraduate, I spent a year on the East coast, and I worked with this composer whose name is Ben Johnston—he’s a composer that studied what’s called microtonal music, it’s kind of like on a piano, every key is a certain frequency, and as an undergrad, I was reading about this in a textbook, I didn’t understand it. So on a whim, I found out that this guy was retired, ‘cause I found out this guy was retired and living in North Carolina, so I literally just cold-called the guy and said, hey, I’m this random guy on the west coast, can I come study with you? And he said, sure. So, I then convinced, somehow (and I don’t know how, in retrospect) the university to give me college credit to go work with this guy. And so they gave me a full year of college credit to go do an independent study with this guy, in part because he was well known, he was not going to [inaud] etc. And then, by pure happenstance, I happen to get a bunch of scholarships like, the day before I left, I would not have been able to make it without them, to drive to North Carolina to go work with this guy on this kind of music—so why am I telling you this? Well, it turns out Ben derived this kind of music that, with a nasty notation system that was really hard to understand. And so I spent a lot of time when I was working with him to try to figure out how to make that easier for musicians, and this led to some led to some albums and stuff like that. I had to involve this like, complex linear algebra translation and all this weird math stuff, but at the end of the day, it had in my head this idea of, you can take a language, whether it be from music or something else, and then you can make that easier for people, and that matters because musicians can’t play it without it being easier. So, when I was in graduate school for computer science now, I had been interested in working with the blind because of my music background, and I just thought, I love doing sound related work. And literally, the first time I sat down with an actual blind student and had them try to use a screen reader for audio, and like they got a compiler error that was a minute long in their screen reader, it was glaringly obvious that the programming languages could be made much easier to use. So it was two things—it was sort of like, working with the musician, that has nothing to do with computer science and having them be a hard notation, and the second was, I sat down with a child that was blind, and they just struggled to use something like C++ or Python or whatever it was at the time, and it became very quickly obvious that if I changed all the words and symbols in the programming language, that it could be easier. I didn’t have any evidence of that at the time, it was just a hypothesis, but since then there’s plenty of data. It’s clear that that is an evidence-based position nowadays. I think that was the first half of your question.
Chris: What is your background, Steffik, how did you become a professor at UNLV and get into computer science and where’d you grow up?
Andrea: I grew up in Vancouver, Washington, which is right near Portland, Oregon. My degrees are in—I have Bachelor of Arts in Music, I did mostly composition while I was there, just sort of writing music, sort of having fun with it. I also have degrees in, Bachelors and Masters and PhD in computer science, and that really came from this sort of music experience: basically, to figure out all these notation problems, I had to do some math I wasn’t familiar with as a musician, and that ended up involving basically finishing a computer science degree to figure it out, and then I just kind of went on from there—the PhD sounded fun, so I went on from there. But for a professor, after graduate school, I really wanted to work with populations with disabilities, and I knew that I could do that in industry. You know, I had a job offer from a screenreader manufacturer pretty quickly after graduate school—
Chris: Which company was it?
Andrea: It was Freedom Scientific.
Chris: You said that that was 2008, so that was after I left, so it wasn’t me who offered you the job.
Andrea: I don’t remember exactly, it was a long time ago, but it sounds like it wasn’t. I mean, they were super kind, they were doing amazing work, but I kind of had this hunch that if I worked at a company, I would be basically making a product, and then that product might get better for people with disabilities. But I sort of had this hunch that if I was an academic, I could do something whacky to try to make a bigger difference for a larger community of people. I’m actually really glad I made that decision, because I feel like that’s come to pass more than I would have expected. So, maybe a little bit lucky, but..
Chris: What got you interested in working with people with disabilities?
Andrea: I have a music background in general, and so that got me really interested in sound applications. And I had a professor that I was working with at Washington State University, and he was interested in music related to coding, which personally I didn’t think would work—and I think the evidence bears that out—however, it seemed like it might be interesting to consider people that might use sound technologies. Who would they be? So I started looking around to figure out who would use a sound-based technology related to programming, like who would that benefit, potentially. And I started having conversations on mailing lists with people that were blind programmers, either in industry or in school, or that were learning, or that were very frustrated trying to figure out how to learn, and stuff like that. And so when I started looking into that, it seemed pretty obvious that there was almost nothing happening in academic scholarship, at companies, to try to make these tools easier for people, and that smelled like an opportunity, and I kind of jumped on it. Since that time, it’s kind of expanded to more disability groups, because why not help as many people as possible, but that was definitely the impetus of it.
Francis: One of the things that we like to talk about on this show is the idea of any reason you can come up with to be optimistic about the future. And I was wondering, I guess, if you are optimistic about the future, and if so, why.
Andrea: I have one that you might be interested in, Francis, just because of your comments about politics and the White House and stuff. So one of the things that we’ve been working on—I’ve been working on—is writing a book related to evidence and computer science, because there’s been this problem where, even in the academic peer-reviewed literature, there’s a death of evidence; meaning there’s a lot of scholars out there that are not using the scientific method and they’re getting published anyway. And that’s unfortunately a fact, there is strong evidence for the position that many scholars are publishing without evidence. However, it also turns out to be the case, that when you look at the history of evidence-gathering in other fields, as other fields change they tend to become stickier with their evidence. So, for example, for a microbiology person you might have a guess, but I’m not sure—do you know when you first, in drugs, had to actually start declaring what was in them, for a company? Any idea?
Francis: Oh wow. No idea, actually.
Chris: Was there a [inaud] in 1996?
Andrea: Oh no, it was early 1900s. Right?
Chris: OK
Andrea: And then when did you first have to test to see that your drugs were safe? Like, when did you, when did a company, by law, have to say, I need to test my drug first to make sure it doesn’t kill people? When did that happen…?
Francis: I imagine that’s a post-“better living through chemistry” affair, so I’m going to say ‘60s.
Andrea: That’s a really good guess, but it’s late ’20s. And then in the ‘60s, it turns out that’s when the laws changed so that you had to gather evidence to determine if a drug worked, at least better than placebo. Right? Which is not a great standard, but it’s better than it was before. And so why am I telling you this? Because it turns out, at each of those small increments, fields lagged the medical sciences started to get a little bit stickier with how they were damping down on pseudo-science and stuff like that. Now, they’re not perfect, we know that there’s anti-vax people today that are nuts and stuff like that, but in the medical communities, a significant portion of these studies that are done by actual scientists, that are published in peer-reviewed venues, follow standardized evidence standards, they follow very complex but also really important checks and balances in procedures to make sure that things work. One of these is called the consort evidence standard. And here’s why I’m hopeful: I’m hopeful because even though fields like computer science might have issues right now—and clearly we have issues politically with pseudo scientists being in charge of federal agencies, that’s a big problem—but I also feel like, we live in a democracy, we get to choose still, and historically it was actually much, much worse for a long time, and even though we have kind of a bad lull right now, a lot of these problems actually have been kind of slowly fixed over time to get rid of some of the nastiness that are there. So maybe in our time, I don’t know exactly what laws need to be changed, but it almost feels like we need some anti-propaganda laws. Maybe we need some laws protecting against megaphones for pseudo-science. And how to balance that with freedom of speech I’m not sure, but I feel like these changes will happen over the course of the next 20, 40, 50 years. And it might take a generation or two, but I’m hopeful that as that occurs, we should see significant changes in technology as the evidence standards improve [inaud]
Francis: You know, I think that regulation is something that we just have to live with in so many areas of life, and regulating accuracy in—it’s tricky, but it’s become something where it’s just too important to not figure out some way to approach it. You know, it’s kind of like we’re in a post-journalistic environment. You know, there was a time when journalism would kind of step in at a certain point and say no, that’s not true, here’s the evidence. And at least there could be a consensus, but now it doesn’t even seem like that’s happening so much.
Andrea Yeah, I think so too. And the nice thing is, too, when it comes to regulating this stuff, it doesn’t have to be any one solution; for example, right now, if you look at cable news channels there’s MSNBC, there’s FOX, and there’s CNN. It doesn’t have to be three, we can break them all up, break them into tiny little pieces, make sure that there isn’t one owner that gets a national news just for himself, and then make it so that, even if there’s one crazy person that has a thing, we don’t have to give them megaphone, right? And the FCC actually used to have laws like this—sorry, not laws but regulations, related to breaking up local groups. Maybe we need to do that. And then it wouldn’t even regulate accuracy, it would just say, well look, you can say whatever you want, you just don’t get a megaphone about it.
Francis: Excellent, yeah.
Andrea: And so, there’s lots of options.
Chris: Is there anything you’d especially, whether it’s yours or somebody else’s work, that you’d like to promote or plug or make sure our listeners leave knowing?
Andrea: Sure. I’ll plug Quorum Language, which is a programming language, it’s often great for schools, it’s highly accessible to people with disabilities but also it’s fun for making games and stuff like that. You can get it quorumlangugage.com
Francis: Great
Chris: Excellent. Well, thanks so much for coming on Making Better.
Andrea: Thanks guys, I really appreciate the conversation.
(music) We’d love to know what you think of our podcast. Please visit us online at MakingBetterPod.com and if you feel like supporting us, leave us a review or rating in Apple Podcasts or wherever you listen to us, or send us a donation. You can find the form for that on our website. Follow us on Twitter @MakingBetterPod. You can also interact with us on Facebook, just log into your Facebook account and search for “Making Better”
—END