AI Agent Guidelines for CS336 at Stanford
106 points by prakashqwerty 2 hours ago | 49 comments

andersmurphy 20 minutes ago
Seems like a pretty close copy of Carson's (of HTMX fame) agent.md from 5 months ago

https://gist.github.com/1cg/a6c6f2276a1fe5ee172282580a44a7ac

reply
xydac 25 minutes ago
This is a very good baseline for future courses to build on, there would always be a group that wants to jailbreak this and thats okay, but have baseline agent support learning is needed in this ai first world.
reply
ezfe 16 minutes ago
Jailbreaking isn’t even needed - you can just modify the file
reply
NickNaraghi 2 hours ago
This would be an interesting approach if the course supplied a custom Harness (perhaps in place of a textbook) and this was part of the instruction set inside of it. As a standalone thing you ask students to import into their agent, seems unlikely to work.
reply
cush 28 minutes ago
This is such a realistic balance between completely banning coding agents and embracing the spirit of higher education
reply
ohmahjong 2 hours ago
This seems somewhat sensible to me - the genie _is_ out of the bottle, and students absolutely will use AI agents to finish assignments without learning a thing, but there is some value to showing how agents can be used as teaching tools and what healthy use _can_ look like
reply
asdff 25 minutes ago
Same issue as with cliffnotes. Easy way out means the easy way will be taken. Unless, you actually design a decent assignment or exam. In person essays or exams, heavily weighted, you are simply screwed if you didn't study the old fashioned way. A couple of my more serious classes were like this: no homework, no projects, entire grade based on 3 exams. That put the fear of whatever diety you subscribe to into you like nothing else to study hard and not fall behind. One bad exam you can't really come back from. Better luck next year when you retake it. Or, you dig in like hell.
reply
llbbdd 57 minutes ago
Agreed. I don't know how they plan to enforce this but this is way better than some other articles that have come up indicating educational bans on AI use, in-person proctoring, verbal assessments, pen and paper exams etc. This is the first attempt at an approach I've seen that doesn't seek to isolate education from reality; students that are effective at integrating AI into their work and actually understand what they're doing are going to get jobs, which is ultimately the goal of school.
reply
JohnMakin 47 minutes ago
They're only cheating themselves in a world that increasingly cares about knowledge (market trend of seniors being preferable hires to fresh out of school juniors) and not the piece of paper that "proved" you had such knowledge.
reply
simonw 2 hours ago
Hah, I like that these are presented as a CLAUDE.md.

(They have the same content duplicated in an AGENTS.md as well - I really wish Anthropic would hurry up and teach Claude Code to check for that file too.)

reply
israrkhan 56 minutes ago
We symlink AGENTS.md and CLAUDE.md to a single file in our repo
reply
cpeterso 33 minutes ago
You can also include other md files like AGENTS.md in CLAUDE.md:

  @AGENTS.md
reply
bakugo 43 minutes ago
They won't, because forcing the file to be named after their product is an intentional marketing choice. Free advertising on every repo that has it.
reply
matltc 2 hours ago
I wouldn't hold my breath.
reply
sgirard 2 hours ago
This is interesting. I don't know how the AI agent guidelines will be enforced because there will always be a model outside the curriculum that a student can use to bypass the guidelines. Encouraging academic integrity is useful but requires the student to buy into the idea that they are paying for an education, not a diploma. This is a tough problem and I have been wondering how CS departments are incorporating AI into the curriculum while encouraging appropriate use in a learning environment.
reply
earthnail 44 minutes ago
Stanford has an honour code. Meant no oversight even during exams. Worked surprisingly well when I was there. The flipside is, if you’re ever caught cheating, there are no second chances.

I imagine this applies here, too, if they want to enforce it strictly.

reply
asdff 29 minutes ago
>Worked surprisingly well when I was there.

How could you tell? I proctored. People cheat pretty frequently and other students are none the wiser. It really takes like 4 proctors if you want to do it right. Even then I'm sure the clever ones are slipping through. These were scantron though. Short response/essay format you'd be screwed if you didn't know your stuff.

reply
henry2023 26 minutes ago
Marc Tessier-Lavigne was Stanford's president from 2016 to 2023. Not sure if the honor code means anything nowadays.
reply
shimman 8 minutes ago
You mean it worked well for cheaters right? The more I learn about these "honor codes" the more I realize how sheltered these American elites have become.
reply
itopaloglu83 55 minutes ago
Well, no amount of instructions would work if the student has no intention to learn anything.
reply
gchamonlive 54 minutes ago
In an ideal world guidelines should be suggestions for those willing to make the best of the course and improve as a person and professional. However a degree has real world value and repercussions, so enabling someone incompetent to do a dangerous job can put innocent lives in jeopardy. It's tough, but I hope in time we learn how to live with this new tech.
reply
rossant 30 minutes ago
Interesting. It makes me think of the idea of fighting piracy by providing a solid legal alternative through streaming platforms, etc.
reply
soldeace 25 minutes ago
I'm definitely going to use a variation of it for learning new programming languages.
reply
recursivedoubts 58 minutes ago
I think these are based on the one I posted a while back:

https://gist.github.com/1cg/a6c6f2276a1fe5ee172282580a44a7ac

reply
brunborg 15 minutes ago
Yes absolutely! We linked your version inside the extended AI policy document, but forgot to add it to our website cs336.stanford.edu
reply
ritzaco 2 hours ago
yeah I don't think that's going to work - it would be kind of like "we're releasing model answers to all assignments but please only use them as a teaching aid and don't copy from them"

best to

a) adapt assignments so that agents are bad at producing solutions

b) have more scenarios where students have to do things in controlled environments. Universities managed to adapt to 'any solution you need is readily available online' so I don't think it will be that different to have several times a month/year where students have to go into a room with nothing but pencil and paper to prove what knowledge they have vs what they have the skills to access

reply
harikb 60 minutes ago
Laptop without internet access, sure. Pencil and paper? that is brutal :)
reply
artificialLimbs 52 minutes ago
I did most of my CS class tests this way within the last year. It’s not that bad because prof doesn’t care about syntax so much (unless that’s what we’re testing on of course) and details, but wanting instead to make sure we understand broader concepts.
reply
jastanton 32 minutes ago
I agree it's not a complete solution. But as those don't exist as a society we are looking for a step function in the right direction. and IMO this is one such step. You may disagree that it's not a very large step, but I would argue it's still in the right direction therefore it is neccesary, especially in education space, and I'm happy to see someone publishing at attempt.
reply
gaiagraphia 35 minutes ago
Is this all an elite educational institution with about $50bil in assets could muster, lol? This is completely and utterly unenforceable, and such, worthless.

There really needs to be diversity in delivery styles for different modules of courses according to their aims, with 'ai access' as a key variable.

If AI is allowed, it should be based on $x of usage/student, with an audit trail to prove no external funding was used, and module aims based on using AI to the max while conserving token use. Like actually creating wild, ambitious shit which takes cutting edge services to the max.

If AI is not allowed for a module, then it really needs to go back to the old skool, with handwritten exams, or coding using old machines and textbooks. Some skills, techniques, etc, really do need drilling.

Straddling the middle will help nobody, result in accusations, increase the burden on teaching staff, and result in a course without a realistic focus.

Though I guess if you're a big brand university, you don't really need to care about innovating. The money will keep pouring in. The whole further education sector is in dire need of a shake up.

reply
chalupa-supreme 21 minutes ago
I don’t really know why this is getting downvoted. It’s clear that higher education is degrading because of easy to reach AI solutions that have no type of penalties for use.

During my undergrad it was normal to see people refer to Chegg solutions to get their answers, or as a friend for theirs.

Maybe there’s a reason my first CS professor wrote out Java code with pencil and paper I guess.

reply
georgemcbay 2 hours ago
> What AI Agents SHOULD NOT Do

> * Run bash commands

Students who prefer to use zsh keep winning.

reply
farmeroy 36 minutes ago
I really like this. I'm currently doing a part time BSc and my current module explicitly allows AI usage as long as you 'cite it'. The guidelines are out of date in that they assume you are using a chatbot and not a coding harness. The temptation to have claude write all my pandas code has become too difficult for my self control, but at the same time I actively feel my education is suffering from using it. As I write my final paper I am thankful that I at least despise AI writing too much to use it for the actual marked assessment, but I still feel that I have cheated myself out of part of my education and probably wasted a lot of time going fast in the wrong direction because generating data frames, graphs, statistics, etc. is just so easy with claude
reply
xyzal 18 minutes ago
I am really baffled by the comments in the spirit of "this is unenforceable, and therefore worthless".

I bet most people would not steal even if they knew they could get away with it.

reply
gaiagraphia 6 minutes ago
Students are struggling to get work after graduating because they're dropped into a competitive environment. Ideals aren't enough to get jobs in the current environment.

Universities should be places which are at the bleeding edge of development and providing society with the best new ideas/tech, etc has to offer. Junior workers should be hotbeds of exciting talent which have the ability to revolutionise industries.

By creating such milquetoast environments to study in, which are seemingly scared or unable to prepare people for the future, students are being done a disservice.

Far too many people are far too comfortable with their cushty positions, and it's not doing the youth any favours.

reply
CamperBob2 16 minutes ago
I mean, some would say that's how this whole thing got started.
reply
cute_boi 2 hours ago
And, yes students are going to follow it....
reply
ChrisArchitect 40 minutes ago
Related:

CS336: Language Modeling from Scratch

https://news.ycombinator.com/item?id=48357075

reply
echelon 2 hours ago
This is ridiculous. The genie is not going to go back into the bottle. This is the equivalent of "you wouldn't download a car". (Yes, we would.)

The solution is to scale the difficulty of the objective measures. Expect far more from students.

Reorient the university around physical laboratories and timesharing resources no single student could afford. It's already like this in many STEM disciplines.

More internships, more networking, more large projects. Less trivial tests of knowledge and credentialism.

reply
overfits-ai 2 hours ago
[flagged]
reply
behnamoh 2 hours ago
[dead]
reply
mi_lk 2 hours ago
good intention but useless let's be real
reply
LVB 2 hours ago
Seeing my own kids (teens) go through some of this, I'm becoming slightly less pessimistic as it all shakes out. Among their peer groups there does seem to be an opinion forming that sure, anyone can just ask ChatGPT for quick answers on assignments, but actually knowing stuff is a bit of a "flex" that's respected.
reply
datakan 4 minutes ago
300 years ago when I was in high school I had a friend choose to go the HVAC trade school route instead of college. He chose the hardest school in the country where they did most things manually so that students understood how things work. It removed the "magic" some tools provide. I was pretty impressed he was wise enough to do that. He's exceptional at his job by the way.

I think we have a tendency to think the worst of your people. They frequently surprise me though.

reply
neutronicus 9 minutes ago
Teens also fucking hate AI, on a cultural-ideological level.
reply
bdangubic 6 minutes ago
I think they may hate what it may be doing to their future outlooks but they use it as much as they do social media
reply
xiaoyu2006 2 hours ago
I always wonder why there is such course. Using agent ai coding tool is trivial.
reply
walrusted 5 minutes ago
using a coding tool is trivial, correct. so is using a microwave oven. but Michelin-level chefs will manage to make star food with or without a microwave, while nobody else can. Because not on that level.. I trust Stanford on this one to bring people up to such a level.
reply
hn_throwaway_99 2 hours ago
When calorie dense food and gas powered vehicles came on the scene, humans (generally) got fat and out of shape. "Why eat that salad and go for a run?" one might say, "This cheesecake tastes much better and I can just drive wherever I want to go."

Getting fat is one thing, but getting stupid is another, and I really fear for the future of humanity when it becomes so easy to sidestep the processes that let us actually learn and grow because stuff like "using agent ai coding is trivial".

reply
gaiagraphia 32 minutes ago
There's different skills at play, and they're both as valuable as each other.

They shouldn't be thrown into a big soup with shaky aims.

We still - as a society - manage to have PE and driving as different subjects. The same can equally apply here.

reply
lukeigel 18 minutes ago
Pangram reports as 100% AI generated. Makes sense for a README, but a tad bit funny given that their students must hand-write code
reply