Alright, so the place I work is transitioning to a new system for grading online exams. I learn a couple days ago that I've been re-assigned as the lead instructor for a pre-calculus course, when I was intended to be the back-up to one of the more senior instructors.
To prepare for this duty, I decide to take some of the exams blind to make sure I actually remember how to do all the stuff in the course. It's mildly unsettling when I get a B on the second exam in the course.
I look over the two problems I got wrong, and it turns out that, no, I'm prettttttttty sure I knew how to do those things. I had a hard time getting up that day so I was a bit tired... who knows, I might be delirious and just may be bad at math today. I throw the necessary equations in Wolfram Alpha and discover that, no, I'm not too tired to do math, it's just the tests are written by drunk MBAs who forgot how to do pre-calc because, you know, that's the sort of thing you don't need to remember how to do when daddy has a job writing exam problems lined up for you out of college. I mean, how hard can math be it's not like anyone's ever held back in high school because they can't do it or anything sheesh
I then learn that, because of the way the hierarchy of the online exam program works, as a not-exactly-senior member of the instructional staff I don't have privileges to open the exam question bank and fix the problems. In fact, only one math instructor has these rights. I bring up my discovery to her today.
She laughs and tells many stories about how the last online grading program (owned by the same company, incidentally) had a calculus course where, initially, roughly 1/4th the questions were flawed, and the support for the site absolutely did not care when informed about this. The best story is when she called the company and was told about the company's vaunted "97% accuracy in writing questions" - which is hunky-dory for those who haven't heard of the binomial distribution, but when you consider that on a 25 question exam, there is a less than 1/2 chance that a student will not get any flawed problems, is sort of a big deal. We bring this up to the person assigned to dealing with this firm. She also laughs. Much disparaging conversation ensues, mainly involving alcohol and the merits of drawing eyes on a parabola as a mnemonic for remembering if the coefficient of the x^2 term is positive or negative (if it looks like the graph is smiling, it has a positive attitude and opens up, and if it looks sad, it's being negative and opens down. Ah ha ha ha!).
My job duties for the rest of the day involve taking more tests and writing down all the problems with wrong answers, which are disturbingly common (yea, change the base of that problem on logarithms mid-problem from 10 in the problem to e in the "right" answer and see if I notice). It depresses me to think of a average high school student who isn't amazing at nor cares about math doing a difficult problem right, having it marked wrong anyways, and suddenly doubting their abilities.
But I mean it's not like we have a staff of people who are fully capable of writing their own math problems for tests or anything. It's, of course, far better to outsource this duty to a company with a proven track record of incompetence.
For those of you on these boards years younger than I... yes, this is pretty much a perfect anecdote to describe how the real world works.