> Ask an examiner from 20 years ago the risk of allowing people to take exams in their own home. They would have said 'cheating', even with no concept of AI.

AI has taken it to the next level. Previously, with many exams you would still have to know how to identify the concepts and related keywords in a word problem to even know what words to look for in the index of the books on hand before you could get to the right page to start cheating.

Some of the certification exams I had to take back in the day even came with their own little reference manual that everyone got and was free to use to look up concepts and equations like you would in the real world. The book wasn’t helpful if you didn’t know how to recognize the way to solve the problem and look it up, though.

AI changes that. Now you don’t need to know anything at all. You don’t even need to parse the question or even speak the same language. Copy the problem into ChatGPT with a prompt attached. Copy the answer into the solution box.

Anecdotally, the rise of ChatGPT has also normalized the concept of cheating among students. The common thinking is that everyone is using ChatGPT, therefore you’ll be left behind if you don’t cheat.

> Ask an examiner from 20 years ago the risk of allowing people to take exams in their own home.

Isn't this like an "open-book" exam? We had them 50 years ago when I was doing my A-levels in the UK, and I always thought it was a good system. The trouble now is of course that you can ask the book to look up the answer, unless the question is very well thought out, which is hard. The open-book thing worked best IMHO for things like practical chemistry, where you needed the technique as well as the theory.

Some exams in the UK (GCSEs) moved away from coursework because of the problem of cheating. If it happens with GCSEs, why would it not happen with high stakes professional exams?

There are some IGCSEs that you can take remotely (with a camera on you and a hefty extra fee) and I am wondering what problems those will run into. Pearson are offering them.

With remote "exams", you don't even know who is taking it.

Who sits in front of the PC, who is nearby?

The rest is kind of besides the point then.

Seriously. Kids are going to cheat. It's already easy enough to just throw the test material into the LLM and get a bunch of flash cards on relevant content and memorize that. I Wish I had AI in college.

It's only a "disgrace" because you still believe the BS at face value. All of these licensing schemes are self (the organization running them) serving rackets to various extents.

Subtract a thousand from all the dates, call it a guild, sprinkle some nobility into the org chart. That'll make it all make sense. Same shit, different day.

Of course they jumped at the chance to charge the same for less. At the time it didn't look like there was a serious downside. "Everyone" was doing it. And to some extent that forced their hand. If you're running a licensing racket and you don't stay up to date with the rest of the licensing rackets and your license becomes relatively a worse value for whatever the upside is then supply will be constrained, prices will go up, perhaps enough to make people not well versed in you trade ask tough questions like "why are these people licensed in this manner" that could be a serious threat to the status quo.

I wonder why it wasn't done earlier as the pandemic has been over for a while.

This was going on long before LLMs.

When I took quantum mechanics in grad school, I struggled through the weekly (and intense) homework sets. My TA was a hardass, I’d spend hours on some problem, several few pages of math work just for one problem, and make some dumb mistake in an integral somewhere, being off by a factor of 2 at the end and only getting 2 of 4 points.

It was painful, and I felt like a dumbass seeing the other kids regularly getting perfect scores.

Then the midterm came and I blew them all out of the water. I hadn’t realised they somehow had the solutions manual so just got perfect scores all along but clearly didn’t learn the material like I did.

80-100% of the grade imo. You could always tell which teachers were serious and teaching serious subjects by how little they cared about your attendance and homework assignments. In math classes, you could tell in an instant when they only assigned the problems that had answers in the back of the book. Not doing your homework in a serious subject is just punishing yourself when the exam comes in and it looks like it's written in a different language.

If you don't do your homework, or show up to class, but you ace the exams, you were just paying for the certification and to me that's totally legitimate.

I went to school with a bunch of working class immigrants who were working full time and going to school full time. If they had to miss every other class because of work but wanted to make up for it by studying all night, that seemed admirable to me. Nothing I hated more than participation points. It reminds me of management desperate to increase their headcount. It's the insistence that the focus of the class is the master-shifu at the front and center. It's a 300-level math class, dude; it's nothing that most people couldn't learn on their own.

having "homework / coursework" count for the final score is what surprised me the most when learning about schooling in the US, in my university 100% of the score was the final, typically written test first, then oral in front of a blackboard (and usually the oral portion could move the needle of the written only +20%, but could definitely have you fail completely).

The one course that had something similar was microelectronics where during Christmas holidays we were given an optional assignment where we could design IIRC a NAND gate (2um process I think, most people ended up with a 5ft x 5ft sheet of paper at the end) which took a long time, but would give you up to +5% at the final (only one person got the full 5%, due to their creative use of the diffusion layer for interconnects). I don't remember any other course having anything along those lines, although to be honest you could slightly influence the difficulty of the oral final questions depending on how hard you worked / your behavior in class (of course only in years 4-5 where courses had only 20-30 students, no chance in year 1-2 with 400+)

It was extremely high stress, as you can imagine, but basically impossible to cheat. Every year a significant percentage of the students had to drop out, so by the time the 5th year thesis came around I think less than 20% of first years graduated at all. You were allowed to retake course finals if you wanted a different score (available 3x year typically, no guarantee you'd do better tho), but if you failed enough times you had to retake the course from scratch. You also were not allowed to enroll in the next year's courses until you passed all the prerequisites.

This comment made me laugh, because I was looking into doing an online highschool, and while looking for discussions on the pros/cons, I stumbled upon the Reddit, which was all "Does anybody have the answers to the Unit 3 test for Mr. ${LAST_NAME}'s MCR3U class? I have $20." or "Selling the answers to the Unit 4 test for Mrs. ${LAST_NAME}'s ENG4U class for $30." That scared me off of doing high school online.

That last sentence reminded me of a story one of my religion teachers told us last year: he was in university, and the way the course was structured was something like there were two exams, mid-term and final, each worth like 50%, but you could choose to not do the mid-term, and have the final be worth 100%. He chose the latter, and ended up being stressed out of the mine preparing for the exam. I can't remember if there was supposed to be a moral to the story, but it was a funny story, at least. It was probably funnier in-person than it will be for the person reading this comment.

I don't know, I've known many people that struggle with exams even if they know the material and even more people that excel with exams that learn nothing. Falling back on any kind of exam is just a recipe for more rote learning and that doesn't create better people (although possibly better readers, which we need).

(Preface: I am not a teacher, and I understand this is a hot take). At the end of the day there's an unwillingness from every level of education (parents, teachers, administrators, school boards, etc) to fight against the assault on intelligence by tech.

I don't think kids should have access to the public internet until they're adults, and certainly should never have it in schools except in controlled environments. Schools could create a private networks of curated sites and software. Parents don't have to give their kids unfettered access to computers. It's entirely in the realm of possibility to use computers and information networks in schools, accessed by children, designed to make it impossible to cheat while maximizing their ability to learn in a safe environment.

We don't build it because we don't want to. Parents don't care enough, teachers are overworked, administrators are inept, and big tech wants to turn them into little consumers who don't have critical thinking and addicted to their software.

I failed one exam in my final year of uni (marginally), but passed the module because of excellent coursework. I put in an order of magnitude more work into the coursework for that class than I did any other class because I knew I was going to struggle in the exam.

In all honesty I shouldn’t have passed that course but it is what it is - and as far as I was (and still am) concerned, it was a bolt on course that I am ok being limited in my knowledge of.

And what kind of skills would you test with this method?

Colleges are clearly not working, as evidenced by the number of unemployed graduates. Some will blame AI, but the reality is that any graduate would require training to be productive in the job, something they didn't learn in college.

My point is, if colleges could adapt to the job market, they wouldn't be in their current state.

In person exams make sense yes, but pen and paper?

If you want high integrity, exams should be done on school computers with extra integrity monitoring software. In the current cheating paradigm, pen and paper are as easy to cheat as students using their own laptops. Note that students can take pictures of exam with their phones, glasses, even their pens. There are pens with cameras and LED screens on them that connect via Wi-Fi to chatgpt.

Oral exams make for the best integrity but they are a pain to grade.

Locks arent meant to prevent doors from being opened, theyre meant to make it harder

If thats the sort of james bond shit that has to be done to cheat on in person exams, then theres a way higher chance of being caught. Using chatgpt at home is way harder to catch

Interesting. I don't recall this happening during my studies. You didn't have the time in my exams to cheat, leaving for the toilet was strongly discouraged and if you did leave, you would have an invigilator standing behind you at the urinal or outside the cubicle.

Just ban bathroom breaks and cap exams two 90-120 minutes tops. If longer examination is needed split it into multiple sessions. I genuinely believe that being able to plan around bathroom breaks or manage discomfort otherwise is a reasonable expectation.

Or you make the tasks smaller and ban bathroom breaks, or replace the task with an equivalently challenging one if break is given. Or you check for phones before the exam. There are a lot of things that can be done.

Certifications have always been irrelevant for me, but that's only because my goal has always been what I'm capable of doing on my own AND (this one is a biggie) I was unbelievably fortunate to have several people in my career who trusted that I could get the job done.

Certifications are about low trust. With the advent of modern LLM tech, trust levels are probably not going up.

Nobody needs to hire someone who can use an LLM because if that is the skill they're looking for they can just use the LLM themselves.

So if you need to hire someone because the LLM isn't cutting it, then you'll by definition need to be hiring someone who isn't using an LLM. Someone who isn't just using an LLM to make you think that they aren't using an LLM.

How is that going to be done? Sounds like a job for certifications to me. Not today's certifications, but a much more in depth, in person, and gatekeepery certification.

My guess would be that certifications, unfortunately, will be significantly more relevant in the days of LLMs. Not less.

I don’t think it will be too long before the pendulum swings back towards “real people who actually know the subject”. At that point, I might feel bad for everyone who coasted on AI.

Get back to me when AI is actually reliably correct about any technical field.

Accounting exams are gatekeeping, yes. The good kind of gatekeeping where you make sure the people doing the job are actually capable. And you have avenues to punish those who fail their clients.

> This also applies to universities

Eh. I’d say the actual academics are about 1/3 of the university experience. The rest is networking and teaching you how to think and solve problems on a more abstract level. I’d say the people who farm that (and particularly the abstract thinking part) out to AI are going to be the ones left at disadvantage in the future. You’re completely replaceable.

I've had no end of problems with accountants regardless of their certifications, they operate in a domain with an incoherent body of contradictory and highly subjective rules yet make it out to be a science.

My conclusion as a whole is that accountancy as a profession rarely delivers any actual value to their customers, where much of the job is compliance theater at best.

Yeah. There was a recent post about this sprt of gaming happening at Stanford

> start using real world examples of what kind of work student would be working with as an employee

College isn't hands on employee training; the employer can do that if they want it. College is for the student, and not just for their career - there is much more to life. Knowledge is power for the student.

Even working with computers, theory is more universally valuable knowledge than the current programming trend - the wonderful thing about a theoretical abstraction is that it applies everywhere. For example, lots of practical high level coding experience is now less valuable, while people who truly understand theory can apply their knowledge somewhere new.

The article is about accountancy. Accountancy is not academia.

>In the end of the day academia in general should stop relying on exams based on memorization of random facts and start using real world examples of what kind of work student would be working with as an employee.

I have an undergraduate degree and PhD in chemistry and I don't really think "blind memorisation" had much to do with my success. It will only get you so far.

I think there is also substantial within academia about the purpose of academia. I think a lot of academics might disagree that it is about preparing people to be employees.

If academia was solely about job training it would take at most like 6 weeks. It is about getting a rounded education. It is why engineers need to take liberal arts classes.

I’m not convinced.

Academia should be for exploration, research, or preservation/archival. It’s about knowledge, not profit.

Academic attainment should be about the subject.

It’s business that should deal with the application, the short cuts, the “ends” rather than the means.

I’m sure there’s a formulation of this which also allows for AI in acedemia. I’m not srguing for that kind of purity. But I am saying that acedemia shouldn’t be treated as the training ground for employment.

What's sad about it? Why should it be censored?

It is incredibly trivial to stick a knife into human flesh.

Triviality is not a dimension of ethics as far as I have come to understand it.