links (95)
Book recommendations, code, code and ai, and game design
I used a modified version of the SASRec model trained with uniformly sampled negatives and gBCE loss. SASRec is pretty much a transformer decoder that operates on item IDs instead of (sub)word tokens. If you want to learn more, I would recommend reading the original SASRec paper along with everything by Aleksandr Petrov and Kirill Khrylchenko. I trained and ran experiments on A100 80GB GPUs using whatever provider was cheapest on Shadeform at the time.
You can tell that you’re in the presence of good design if you have thoughts like “huh, this ended up being easier than I expected”, or “I never have to think about this part of the system, it’s fine”. Paradoxically
The “Dependency Cutout” Workflow Pattern
It is both practically necessary and morally required for you to have a way to temporarily rely on a modified version of an open source dependency, without permanently diverging.
Toy Story was a transitional project. Since Pixar couldn’t send digital data to theaters, every one of the movie’s frames was printed on analog film. When Toy Story originally hit home video, that 35 mm version was its source. Only years later, after technology advanced, did Pixar start doing digital transfers — cutting out the middleman. And Toy Story’s look changed with the era.
No one has ever once asked me to work late into the evening or on a weekend. Have I worked late into the evening or on a weekend? You bet! (I’m writing this very sentence on a Saturday while my 10-month-old is asleep upstairs).
Angel Investors: A Field Guide
Angel investing is a hobby for me, so it’s really about where I get energy and what I want to be thinking about in my limited free time. I discovered I am almost exclusively interested in infra companies where the team is building a product in a B2B SaaS domain I’m familiar with. I particularly like product-led growth and I love developer tools.
PyTorch handles exascale training now. It powers foundation models that are redefining intelligence. It’s in production at virtually every major AI company. It’s taught in classrooms from MIT to rural India. The tools I dreamed about making accessible? They are. The barrier to entry I wanted to lower? It’s almost gone.
Game Design is Simple, Actually
Building toys is hard! Arriving at those rules and constraints to define a nice chewy problem is very challenging. You can think of a toy as a problematic object, a problem that invites you to play with it.
On the other hand, it’s not hard to turn a toy into a game, and people do it all the time. All you have to do is invent a goal. We shouldn’t forget that players do so routinely.
I’m about to rob you of a dopaminergic experience, because agents are so simple we might as well just jump into the code. I’m not even going to bother explaining what an agent is.
<gifs>
