"Not Even Wrong" Podcast
Investing in fundamentally new concepts and engineering practices with large impact.
Book discussion “The Alchemist” by Paulo Coelho. The essence of life is following your mission. Learn to speak the language of nature. “To survive the desert you must speak the language of the desert.”. Humans build complex scaffoldings out of logical anthropomorphisms. That’s dangerous. Paradox of human existence. Learn from nature and think but don’t let your frontal cortex take over. Strike the balance through love. Human potential is a natural thing. We often suppress it. This book is like a condensed version of the bible. Love and pain are inverses. Both are real.
❤️ From Software 3.0 to Software 4.0. Reward shaping is the next hot programming language. We extend Andrej Karpathy’s concept of Software 2.0, Software 3.0 and vibe coding to the next stage. Reinforcement Learning and reward function design. Software 4.0 is when agents iteratively explore configurations of pre-trained weights . It’s one step further in flexibility, adaptability and autonomy. Reinforcement learning is the programming language of agents.
❤️ Book discussion “Angle of Repose” by Wallace Stegner. Masterpiece. Artistically original, historic, philosophical, political, social and human. History is recursive, present determines history and history determines present. Can’t change history but adapt. Human nature doesn’t change. History does. Conquest of the West. Visionaries, waiting on platform for trains that haven’t even been built yet. Sometimes they arrive. West was built by engineers and cultivated by ladies that brought an imagination of civilization with them. Breaching the trust of love is the ultimate Angle of Repose. Angle of Repose, when unrelated particles move together through the arch of time and at some point they stop evolving. What is your Angle of Repose?
Jiajun Wu (Stanford). Product of Experts for Visual Generation. Improve video generation by incorporating priors from physics and semantics from LLMs. Technique: Sample several potential segments for diffusion and discriminate against those that don’t fit the discriminator. Neuro-Symbolic Concepts. Data efficiency. Compositional generalization. Continuous learning. Zero shot transfer. End to end systems require lots of data and don’t generalize well. Concepts are objects (apple), relations (left of) and motion (put apple left of cup). Reduce concepts to atom level. Then build up from there. Compositionally. See presentation of concept learning.
❤️ CVPR 2025. Unify video and actions, enable inverse dynamics. Shuang Li, Shuran Song (Stanford). Unified Video Action Model. In robotics, we are interested in learning generalizable policies that map observations to actions. Train video and action together and represent both in latent space. But use different head for video and for action prediction. Analogy. Dream surfing. See wave, imagine actions and then surf in your mind without seeing the wave. Enables inverse dynamics. Deduce actions from video. Train Optimus with youtube videos.
CVPR 2025 Test time compute and autonomous driving. Centaur: Robust End-to-End Autonomous Driving with Test-Time Training. How to make sure, end to end trained planner is safe? Data driven fall back function. How? Computing a gradient of the network weights to minimize an objective function by using a small dataset collected during deployment. Optimize the network’s weights to reduce planning uncertainty, estimated in an unsupervised manner. Use cluster entropy as signal that the planner is uncertain.
CVPR 2025. Data collection for generalist robots. Dhruv Shah (Princeton, Physical Intelligence). Guiding Data Collection via Factored Scaling Curves. Different data can have different effect on performance. Varying camera pose and different table heights, The combination of factors can have even more impact. Tesla is attempting to solve the data scarcity problem with video learning with internet scale data (youtube). Bitter Lesson.
CVPR 2025. Discussing two papers on simulating traffic environments. 1/2 Building reliable sim driving agents by scaling self-play. Simulate traffic through self play. Reward function design determines behavior. No Ground Truth. The Waymo dataset serves as prompt. No bad data in RL. RL is synthetic data generation engine. 2/2 Self Play. Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments. Take scenes as prompts, translate them into vector space, use diffusion in-painting to create more scenes and scenarios and use these synthetic scenarios as training for agents.
Trump must fix the three D’s, Debt, Deficit and Demographics. 1/3 Demographics. Reinstate rule of law at the border and within the country. Immigration is good. Illegal immigration is a recipe for tragedy of the commons. 2/3 Deficit. Stop spending more money than you bring in. Deficit drives corruption and debt. 3/3 Debt. The US has too much debt and it’s crippling the country. Debt is a matter of national security. Eisenhower military- health -, academic - and financial industrial complex, all driven by debt.
❤️ Critique paper “The Illusion of Thinking”. Scaling limit to reasoning. Nomenclature. Anthropomorphize computers. They “reason”, they “think” etc. No. Their circuits and algorithms are different than humans'. Reasoning in AI means automating Chain of Thought. Reasoning doesn’t scale because it wasn’t trained to do so beyond certain complexity. We need better evals, training. Complexity. Low, enough for LLMs. Middle, where reasoning works better and high, reasoning fails. Complexity is in the eye of the beholder. We humans are sequential thinkers, so complexity is high level of sequential steps. Quantum computers disagree. Way forward is RL, self learning. Bitter Lesson.Search for better ways to compute. Andrew Davison presentation. Hardware lottery.
Trump versus Musk is Deep State versus Maker State. We elected Trump to “drain the swamp”. His most important mandate is to “stop the federal government from wasting money”. As it turns out he doesn’t want to. Musk put in time, money and reputation to fix this. He is a proponent of the Maker State, people who strive to create a better future. The Deep State is all about rent seeking, mortgaging the future for immediate benefits. Trump is not delivering. Paraphrasing Naval: “The Country needs Tech for the future”. Time to create a Maker State.
❤️ AGI is not multimodal. The Bitter Lesson revisited. Bit from It. A true AGI must be general across all domains. How granular should world model be? Physical world that cannot be represented with symbols such as language. Contrary to Wittgenstein and Eco.
Bit from It. I think we have it wrong. We want to understand physics through language, when evolution created the exact opposite, language came from physics. The Bitter Lesson Revisited: Training modalities separately and evaluating them on different, modality specific criteria, will lead to Frankenstein AI. George Konidaris, Robotics is AI, not just an application. “Sensorimotor dilemma: The more complex an agent the less suited the sensor suite to one particular task.”
Hardware lottery. Research depends on available hardware. Sensor lottery. Embodied Ai works through sensors. World model depends on resolution. Goodhart’s law dominates measurement. Find representation that builds cognitive bridge between real world and robot. Generative AI and RL. World Lab.
World Lab. A Fei Fei Li company for foundation models for real world AI. Surge in embodied intelligence comes as a second order effect of generative AI. Foundation models to generate real world scenarios. Robots can learn to act in real world environments through RL. Spatial intelligence. LWM is Large World Model. What is seeing? Moving in space. RL is a synthetic data generation engine because every try and error is part of dataset.
AI risk and social acceptance. Power and wealth to certain players which will cause envy and social distress. Risk of backlash. Ratio of p to q must be above 10. p is value to society and q is value accrued to company’s shareholders. Must be above 10 so that society accepts it. Tesla and Nvidia have ratios much higher than 10. Most companies don’t. For example, Warren Buffett style investing is p/q<1 or even less than zero. Destroying social value to extract value for themselves. Insurance, Coca Cola, even Apple. Predatory and extractionary capitalism vs. wealth creation. Investing is as much about wealth as astronomy is about telescopes.
Generative AI enables robotics. Second order effects of technology. Highways enable suburbs. Social media enables Bitcoin. Generative AI enables robotics. Scenarios for RL. Training at scale. Massive data generation engine. LLM, VLM, VLA, Nvidia DreamGen all developments that root in Generative AI and enable robotic learning at scale. Create scenarios in combination with actuator data. Driving robotics at speed of Jensen’s law.
The biggest revolution of all. Musk is dismantling Auto, Oil and Labor. These three constituencies are blocking much needed political reform in the US. Instead of changing government (DOGE), Musk is changing the situation of the ground. Politicians are just actors reflecting their constituencies. Change the latter and you change politics. Fears of bond market cracking. Why? Deficit. Technology is solution to government sclerosis in US and competition with China. Hard to predict future, but what will not change? More intelligence, more to be done with less.
❤️ ICRA 2025. Everybody is entitled to their own priors, but not their likelihoods. Data driven learning is a necessity. If you don’t take advantage of Jensen’s law, you will be left behind. Bitter pill is the path forward. Certain task require certain priors. Should we build robots based on data driven learning or put guard rails and priors. The industry will move to the former. Foundation model. Reasoning. RL. Safety is typically regulatory capture. End to end driven learning eventually improves safety. Reward shaping for RL is the new hot programming language of robotics. Musk talking about self play, RL based learnings for Optimus. If Optimus can watch videos and learn from video, you have scalability. Self play. Robotics will advance at Jensen’s law pace.
ICRA 2025. Physics based versus learning based modeling. Graph Based Neural Dynamics in the middle. Yunzhu Li, Columbia.
Model physics with Graph Neural Nets. Nvidia Warp incorporates Python with Cuda and GPU accelerated physics simulation. Nvidia DreamGen, image to video simulation. Inverse dynamics. DreamGen leverages human teleoperation into simulated video action models so the robot can learn in RL Problem with Graph Neural Nets is that the messaging between nodes and edges is not clear. Requires more physics and material science knowledge. Can be learned from pixel based representation. Engineering requires priors to make things work. Using priors in the 1600 would have missed Galileo.
ICRA 2025. Do we need human-robot interaction? Why even separating human from robot? Difference? Perpetual robot Turing Test. Is there such a thing as robot intelligence? Why not just intelligence? Is bio inspired robotics the right way? Use bio as bootstrap for RL. Dario Floreano EPL. Uninitialized environment, morphology, neural net for planning.
ICRA 2025. Robotics, Vertical integration and Tesla. Robotics stack. Metal to operating system to application to AI. Vertical integration is key for low cost, scale production. AI appreciated but not clear. Traditional model predictive control versus learning based. You get what you compensate for. Musk wants lots of AI, growth. Compensate him for that. Robotics requires much, much more training and inference infrastructure. Data scarcity in robotics can be solved with video to ego centric combine with actuators. Create lots more data. RL creates huge data stacks (mistakes are synthetic data). Bitter lesson. Data, general models. Vertical integration makes this work. Big changes can only be made with fully integrated companies. Disney, Ford, Amazon, Apple.
ICRA 2025. Panel. Data or model based learning? Priors or learn end to end with raw data? If you had enough data, you could train without priors. Priors are just training data in disguise. Eventually both systems converge. Everything we know is data driven learning. What you can represent, you can transform. Wrong. RL allows for more flexible function approximation. Key is combined vision with actuators. Third party video data, translate into ego centric vision and combine with actuator information means learning at scale. Tesla did this with Chinese FSD. RL is the programming language of robotics. Simulation, Omniverse.
AI is the oil of our time and Jensen Huang is Rockefeller. Energy -> AI. It’s hard to predict what’s going to change. But what is not going to change? More work, for less. AI is going to deliver that. Countries, sovereign AI is more than just marketing. Countries want their own models trained to teach their kids or manage infrastructure. AI is like ports, schools, universities. Control inference.’If you don’t have your own models, you’re at the mercy of others.
❤️ Sergey Levine (UC Berkeley), Foundation models robotics. Anatomy of foundation models for robots. Pre training, not high quality, unsupervised learning. Fine tuning through experts, RL and/or experts. VLM. VLA. Reasoning, test time scaling for robots. CoT for embodied agents. Vladlen Koltun presentation on RL for drone racing. RL + Simulation. Simulation gives RL the power and status it deserves in robotics. VLM + Action expert. Action expert is trained with diffusion model to match flow/action behavior. See paper
❤️Do we need world models or just end to end training? Reasoning depends on different abstractions. Some things are easier to predict, others easier to do. Skiing vs. math.
Reward shaping is the new hot programming language. Reaction to interview on Twiml podcast interview about RL and reasoning. Reward functions. Intelligence is predicated on suitable reward functions. What is a reward function? A function that shapes behavior. Evolution of programming languages; Assembly, Fortran, C++, Python. Then shift to LLM and natural language. RL reward shaping is a native AI language emerging through learning. Bitter Lesson pilled programming. Data curation. Eval. Reward function design. Agents create synthetic data, evaluate and design reward functions. Composability.
Regulatory arbitrage for FSD. Tesla is transforming itself from selling cars to selling miles and labor hours. Align regulators. In order to accelerate regulatory approvals and prevent regulatory capture and other kinds of opposition, Tesla should apply regulatory arbitrage. Go to places like Dubai or Saudi and launch services. Show what’s possible.
Book discussion “The Razor’s Edge” by W. Somerset Maugham. 1. The life of a person is their path. Success is finding your own path. Applies to business, arts and life in general. 2. Larry is an early ‘Catcher in the Rye’. 3. Happiness is finding your own path. Convention is in the eye of the beholder. Virtue is going your own path. ‘Be original, at all cost’. 4. America on the eve of world dominance both politically, economically and culturally. The land of impersonated achievements, hero culture. Interesting to contrast early 20th century view of America with early 21st century. Today hero culture in jeopardy. Strength of America is living in contradiction (strength - kindness, freedom - ideology, money - waste, pragmatism- tenderness).
Humanities and AI. AI saves humanities and humanities save AI. Fundamental questions haven’t been answered for millennia. Why now? Because we can. Computational irreversibility - Stephen Wolfram. Some things you just can’t think through because of complexity. Must iterate. Increasing compute enables better approximation of answers to big questions such as “what is the right thing to do”. Humanities is about humans. But limited computational ability. AI can help us find better answers and ask better questions.
Reaction to Redefining Energy podcast episode. Wind. Problem with renewables is that they mingled with Woke ideology. Left wing extremism has nothing to do with energy independence and renewables is all about energy independence. China is leader not because of environment or Woke ideology but because they want energy independence. Electrifying transport is key . Tesla is leader. Woke ideology is against Tesla, which shows hypocrisy. Renewables must un-entangle from Woke. “Wind needs cables, solar batteries.”
❤️ Discussing lecture “From Homer to Gutenberg” at Ralston College. Evolution of knowledge and its proliferation from narration to written word, verse, prose and printing press. Language is a technology to communicate knowledge. Language came before speech. Greek is the technology to transcend millennia of knowledge. Printing press was important but translating Latin to vernacular made even bigger impact. Today, LLMs are translating incomprehensible scientific language to vernacular. Large impact.
Book discussion “Somebody’s Fool” by Richard Russo. Average American life intertwined with contemporary culture. Opportunity, love, race and fairness. Predicable. Narrative flows smoothly carried by Russo’s crafty slang. True character is what they know, not what they know about. Real life doesn’t care about how we feel. But we do influence events. Society happens where emotions and reality cross. “Are gifted and talented kids being given a gift?” No. Legitimate questions about opportunity have been taken out of context in politics. Woke is what happens when good ideas get hijacked by bad people.
Manufacturing in US. Not brining back. Reinventing. Trump creating new wave of manufacturing. Best hires are PhDs who can win a bar fight. 1/4 Software defines manufacturing 2/4 Software defines product 3/4 Culture - accountability, ownership, iteration at scale 4/4 Nvidia Omniverse is the CUDA of today. Manufacturing in virtue.
US manufacturing lineage. Edison, Ford, Disney,Jobs, Bezos, Musk, Luckey. Anduril. Article in Arena magazine about Brian Schimpf. Jensen speech in White House. Nvidia is building infrastructure for software defined manufacturing. Ownership is key. "Pouring government money into broken systems and broken cultures doesn’t unbreak them.”
Ownership. Will. Agency. Accountability. Risk. Low cost of failure and error correction.
Renaissance Now. Truth. Beauty. Discussing Routable Symposium at Ralston College. Is there objective truth? Yes. Truth is necessary for error correction. Same with beauty. Post war Relativism and its contemporary cousin Woke are truth deniers. Replacing truth with power and thus destroying society. End game is Huxleyian dystopia. Goal of Ralston is to revive truth seeking and proliferation of beauty in science, arts and music. Important. “If God communicates through beauty, technology is how we respond.”
Chronopunk: A novel. Epilogue. Started writing in 2024. Inspired by current events. Western society derailing into Orwellian dystopia. Woke virus, censorship, political infighting driven by fiscal profligacy. Chronopunk is attempt show what went wrong and how to fix it. Trump improved things but Woke virus still lingering. Nevertheless, we are hopeful. Most important message of the novel is that liberty and knowledge creation must be preserved. If we give up on truth we have nothing.
Tesla Robotaxi business 300$/share, driven by manufacturing and lower cost per mile. Paper discussion, self play RL + behavioral cloying.
Tesla launching Robotaxi. Main driver of earnings is manufacturing at scale, low cost per mile. Autonomy is key driver for now, but longterm it’s manufacturing. Today Ashock, future Lars and Franz. Cost per mile are driven by cost of production, maintenance, energy divided by miles driven. Miles driven main reducer of cost per mile. Autonomy will be commoditized by some sort of AI out of left field, kind of like ChatGPT for robotics.Tesla advantage because it can scale at low cost.
Paper discussion: Robust Autonomy from Self-Play. Real world AI will best be achieved by self play RL with real world priors from behavioral cloning. Sim to Real. Perception.
Chronopunk: A novel (Episode 16).
“We chose to bring an idea back in time, not a technology.” Mody answers ; ‘why did you choose to bring Bitcoin back from the future?’ Sergey Levirov, a Berkley scientist, is chosen to help Mody launch Bitcoin. Sound money safeguards liberty. That’s an idea with preserving. Technology requires humility. Is Chronopunk a Faustian bargain? Yes, but good. “Sound money is inherently anti-elite. It strips power from the state and places it directly in the hands of the people. But it’s subtler than that. It reveals hidden truths — like a search algorithm fine-tuned to uncover clues about objective reality.”
Chronopunk: A novel (Episode 15).
“The only knowledge worth preserving for any generation—past, present, or future—is the ability to keep creating knowledge.” Lisa makes the case for why Mody should launch Bitcoin. Jeff Swayne, the Fed lawyer, defends Ben Bernanke’s actions. The court sides with the Fed. Go to Supreme Court. Mody faces risk of having to stick around to launch plan B, which is Bitcoin. Money and liberty are two sides of the same coin.
Paper discussion. 8. World Model. Create a world model that accurately represents reality. World model = give the robot understanding of how to move in space. RL used for planning. Key contributions: Physics based rigid body vectors, Neural ODE dynamics, Eval of world model directly in virtual space, Sim to Real generalizable.
Chronopunk: A novel (Episode 14). Mody launches Bitcoin. Plan B; if Supreme Court doesn’t uphold lawsuit against Fed, launch Bitcoin. Technical discussion about how quantum entanglement could solve Byzantine General’s Problem. Gen. Douglas is upset because Lisa wants to reveal Mody’s identity. Hard money takes care of deficits, trade imbalances, inequality and fosters progress.
❤️ Anatomy of AI. Anthropic; inner workings of LLMs? Are we anthropomorphizing AI? Knowledge circuits. Map how knowledge is stored in neural net. Sparse programming. Train a model that “upscales” AI models. Map features that are represented in knowledge circuits. Attribution graph. Language is universal: shared grammatical mechanisms across languages.” Similar to David Deutsch: Language before speech or Knausgaard’s hints at Biosemiotics. Whether trees, animals or humans, all living beings have language, some have speech. Unfaithful reasoning.
Bitter lesson pill. Don’t try to mimic how humans learn. CoT is not always what the model actually does. Models are trained with RLHF, trying to satisfy humans. Humans often reason without grounding, working backwards from an argument to reason.Politics, Wall Street. How does this affect robotics? FSD? FSD is trained on human driving. It drives like humans. Not like machines.
Chronopunk: A novel (Episode 13). Deborah before Supreme Court. Habeas Corpus. Demarcation line between state and individual. Property rights extended to future generations. Fifth Amendment. Society would be better if taking future generations into account when making decisions. Fairness. Mody must make a big decision. Stay or go. Is there a plan B? What if the Supreme Court upholds the Fed’s QE? Choice between two women. Lisa is devastated. Or is she? They have a son. She talks to him and leaves traces of hope and hints resolution for why Mody stayed back.
Trade + Lithium refining + 4680 battery = Perfect Tesla storm. Teslas vertical integration is competitive force. Less low cost supply from China, more domestic production of refined lithium and 4680 will drive margins. Designed for longevity, low cost and high return on capital (lower cost per mile). Lithium refinery in Texas is crucial. 1 Billion investment. Access to 50 Gwh of refined lithium which is good for roughly 750k cars. Acid free process. Cost savings could add 5% in gross margin. Add 4680 gross margin expansion by 7-10% in conjunction with trade policy. Applied to 1/3 of Tesla’s production yields 2-3% gross margin expansion for corporate.
How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training.
Key findings: 1/2 Knowledge builds best on existing knowledge. 2/2 First build circuits then optimize. Case for foundation and fine tune. What is a knowledge circuit?
-
specific patterns of connections and weights within an LLM
-
Formation
-
Optimization
Knowledge is a function of compute and/or energy.Takeaways:
-
Create a large set of synthetic data with lots of diverse scenarios, build as many circuits as possible. Then fine tune on real, practical tasks. Simulation.
-
Strength builds on strength.
-
Control for forgetting.
-
Visualize circuit formation during pre-training. Debug knowledge formation in real time.
-
Adapt training dynamics to circuit evolution. First broader data sets then more fine tuned. Curriculum training.
-
Knowledge circuits and inference circuits.
Chronopunk. A novel (Episode 12). Supreme Court. Deborah interrogating Bernanke. Bernanke talks with his eyes, his mouth is full of double-talk. Mody is explaining Moral Hazard and the technicalities of the perfidious debt monetization scheme by the Fed. Wrapping economic concepts such as Moral Hazard, debt and the social impact of a deficit culture into an engaging narrative. Economic concepts accessible to generate audience. Moral Hazard - pernicious actions start small and then grow to big problem. Orwell. 1984 was not supposed to be manual. It is a novel.
Book discussion “Wolves of Eternity” by Karl Ove Knausgaard. 1/4 Epistemology. Our science is good at determining what isn’t. But what about explaining? Shakespeare book analogy. Do we understand our reality? What about other avenues, like semiotics, mystics? 2/4 Judgement is human trait. Penetrates lives and science. 3/4 Existential novel. Small things add up to big picture. You are what you do, not what you think you are. 4/4 Music weaves through novel. Describes characters, ads emotions to narrative.
Chronopunk. A novel (Episode 11). Philosophical and scientific backbone of the novel. 1/3 Complexity arises from initial conditions and update rules. 2/3 Everything that is possible actually happens. Hugh Everett. Quantum computers visualize multiple trajectories. A trajectory is scrambling and unscrambling an egg under constraint of energy. Quantum theory is the theory of what is possible. Reality is a fact. Randomness an anthropomorphism. Choice is between trajectories. Morals and truth are embedded in physics. It’s the trajectory with most optimal energy. 3/3 Reconcile Gödel with Everett.. Truth is embedded in physics. Proof requires an outside view on trajectory to decide that this is better than that.
Spirituality and Science are two sides of same coin. Get to similar conclusions because goal is the same - learn the language of nature.
Tariffs, markets, Tesla. Trump creates chaos with intention to negotiate bilateral trade agreements. Goal is to restore fairness across major trading partners. Risk is at home, Congress, activist judges. In order to push agenda, Trump needs tariffs, lower government deficit and real growth through less regulation, Doge and lower taxes. This requires work with Congress. Longterm positives: Government efficiency. Innovation, AI, tech, robotics, crypto. Tesla deliveries low due to Y transition. Committed to Tesla investment. Manufacturing, lowest cost per mile. Three vectors 1/3 Cost of production, 2/3 Higher utilization and 3/3 Lower cost of operation. Same for Optimus. Key to success will be lower cost.
Waymo interview with Drago Anguelov. We're comparing Waymo to Tesla FSD. Modular versus end to end. Passing gradients through model. Global optimization versus modular. Two problems of self driving: 1/2 Develop driver 2/2 Test driver (what does it mean to be a good driver?). Tesla versus Waymo. Waymo modular, Tesla end to end. Waymo lots of sensors - complexity. Tesla only cameras - less complexity. Tesla hardware/manufacturing optimizing for miles driven and cost per mile. Lowest cost per mile. Drago says that evaluation of driver is hard (what is ‘good’ driving? Reward function?). Tesla has internet of driving. Key advantage. Summary: Tesla 1/4 Cost per mile 2/4 Reward function 3/4 Lower operating cost, less depreciation, lower maintenance - less complexity 4/4 Anywhere to anywhere, harder in short-term but longterm winner, like Google versus Yahoo.Suggestion: Function calling to deal edge cases for bespoke solutions.
Why academia is wrong and what’s wrong with academia. 1/4 Too much funding by government. Move research funding to private sector. 2/4 Teaching and research are mixed. Main purpose of academia is to teach next generation of engineers and business people. 3/4 Academia overestimates its impact on progress. Real innovation happens in enterprises. Entrepreneurs are the drivers of progress. 4/4 Not enough free thinking in academia. Derisive comments about corporate achievements such as Tesla FSD, Nvidia etc. exhibit petty behavior and often contempt. The universities have lost us. Elite blob detached from real economy.
Chronopunk: A novel (Episode 10) Deborah
interrogates Ben Bernanke in court. It’s time to discuss the key concept of the novel; does the government have the right to confiscate property without due process and compensation? No. Fifth and Fourteenth Amendment. Quantitive Easing is consisting wealth of future generations to satisfy short-term goals. Hedonistic culture of the West. Instant gratification versus longterm planning, family, future generations.
Niagara: Single camera 3D image reconstruction. Monocular image reconstruction. Key for self driving, robotics and drones. Camera only solution. Problem. When only one camera is given because of obstruction, glaring etc. then algorithmic solution can help scene reconstruction. Robot must understand edges. Edges often get blurred in 3D scene reconstruction. Solution. 1/2 Represent point cloud as field: Gaussians plus normals. 2/2 Smooth field into a 3D reconstruction by predicting the edges with self attention and constraints on how transformation happens (afine). Self driving completion is a function of compute. More compute will improve camera based robotics. More data, better algorithms. Test time scaling produces more data for pre-training.
Chronopunk: A novel (Episode 9) Three important factors 1/3 Dichotomy between the professional and artistic. Different trajectories, more nuanced personalities. 2/3 What exactly are we protecting, when we say “protect the rights of future generations?”. DNA. Habeas Corpus. Confiscating property of future generations is akin to confiscating property of present individual. A person is a continuation through time, connected through DNA. 3/3 Entertainment is best outcome. Engagement.
Chronopunk: A novel (Episode 8) What does it mean to have a present if you control the past? Senator Von asks crucial question: how do you prevent eternal Moral Hazard? Politics becomes spiritual because we are opening the ‘gates to infinity’. Everything under the laws of physics is possible, since you can always change the present by altering the past. Atifa Benkader, his counterpart in the Stop QE senate committee, talks about the purpose of the mission - compares US to drug addict. Dichotomy of professional and artistic persona. Two strands in novel. 1/2 Economic (QE reason for US deterioration) 2/2 Philosophical (what' the meaning of present?)
Chronopunk: A novel (Episode 7). Getting to the meat of the novel. Explaining economic landscape prior to 2008 financial crisis and reasons why the Fed chose to print money to bail out Wall Street. That’s reason why Mody is sent back in time to prevent this from happening. Key issue is ‘Moral Hazard’. Mody connects with Deborah Fidler, high finance lawyer, to fight Fed. First intergenerational contact via Neuralink. ‘Small thought for me, big leap for mankind’. Chicago blues clubs.
DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding
Why do models do what they do? What is their reasoning? Semantic understanding for robots (child, tree, car. What is their intent?). Helps understand and trouble shoot. Dataset, benchmark for reasoning in self driving. Do we need heuristics? “Priors are training data in disguise.” Robots are embodied LLM interfaces. Accelerate robot learning with LLMs. Create evals for reasoning. Visual reasoning extends the capability of LLMs such as Chain of Thought by requiring models to analyze complex visual data and construct logical step-by-step interpretations. LLMs can help improve simulation.
Chronopunk: A novel (Episode 6) Lots of action. Sometimes nothing happens in years and then years happen in days. We learn about Mody’s parents, his mother’s fate and why he grew up in Sydney. Mounir chooses Iceland as an escape. Manifestation of time as visible in tectonic formations which shift dynamically. The actions slows down towards the end of the episode. Mody chooses MMA as a career because it’s the perfect mix of action and intellect. Excellence. Eventually he follows his father’s footsteps, both athletically and intellectually.
Reaction to Optimus analysis by Connect the Dots. Optimus will succeed where human intelligence is needed but humans don’t want, can’t or should not be present. Tesla advantage: 1/4 FSD 2/4 Data collection 3/4 integrated manufacturing 4/4 scale. Tesla problems 1/2 CEO not present 2/2 must iterate more on hardware, software, AI, integration and business model. Fundamentally the humanoid robot race will be won like any other hardware race through integration and rapid iteration of software, hardware, AI , scale manufacturing and network effects. Tesla excels on all these vectors.
Lanz & Precht Reaction. Thoughts from Silicon Valley. Tech Faschismus? Young people firing bureaucrats. It’s the young who suffer most. Covid. Debt. Trump is giving them their future back. Musk is catalyst, needs efficient government to compete with China and go to Mars. Western democracy relies on bureaucracy executing will of elected officials. Today, bureaucracy is an AI gone wrong. Doge is fixing that. Prevent arbitrary behavior of power. Fourth branch of government is unchecked. 92% of DC vote Democrat. Something is wrong. 40 Bio. $ debt is result of unchecked power. Regulation has been politicized. Example, Space X denied lift off because of political views of CEO. Academic industrial complex financed by government undermining knowledge creation. Tuition way too high. Risk of Trump. Mafia democracy. Loyalty. US has history with that, George Washington. Democracy is the system where power changes hands without violence.
Institutions are the life blood of democracy. Yes, but what if they don’t work. Like congress? Trump, Milei (Argentina). Risky, but necessary. Vance said: “Protect our values, not protect our tech giants.”
Chronopunk: A novel (Episode 5). Simbi and Mounir get together. Mounir excels both in academics as well as in MMA. Their relationship if full of contradiction, tension and romance. It’s anit-fragile. Questions about physics, science. What is precision? How much details do models need to be able to reproduce reality. Time? Definition of life from point of view of trajectory modeling. What’s the difference between a rock and an eucaryote? The former’s trajectory can be modeled versus the latter is subject to random transformations. Or îș it?
Neurips 2024 Part 17. QueST: Self-Supervised Skill Abstractions for Learning Continuous Control.. Garg et. al
Design architectures that lend themselves to efficient cross-task transfer. Discretizing action space allows for GPT style transformer architecture. Autocomplete tool. Represent robot motion in terms of codebook. Quantize motion into discrete sections of space and time. Data collected via teleoperation or scripted robot action. 1/3 Learn code book from teleoperation. 2/3 Map RGB pixels and actuator data to codebook. 3/3 Inference: Zero shot autocomplete from prompts.
Elon’s move 37. Doge is about streamlining US government to compete with China. Srategic to Musks’ missions. On first sight Doge might look destructive and a waste of time. But it’s not. History of tech and wealth creation is full of move 37 moments. Tesla stock down because of macro (fears of recession) and specific company problems such as Model Y transition and Chinese competition. Backlash not material. Domestic terrorism is. Trump declares attacks against Tesla domestic terrorism - prosecute people who fund this violence. In times like this focus on fundamentals. FSD, Cybercab, Semi, low cost model, Energy, Optimus. High earnings potential through FSD unsupervised and Cybercab. Miles on demand. Tesla should design cyber cab network solution in cities like Dubai. Technological core (Christensen) that allows disruption across transportation and energy value chain.
Chronopunk: a Novel (Episode 4). Romance and conception. How did parents meet before they conceived a child? Matters for intergenerational time travel. Mody meets Lisa. Mounir meets Simbi. Music, feelings matter when conveying a narrative. Bristol music system, trip hop. Excellence, risk and agency.
Thesis discussion: “Learning in the Quantum Universe” by Hsin-Yuan Huang. Pivotal paper. Good example how science should work. Bridge quantum with classical. 1/3 Quantum information can be formulated into classical compute. Models can be learned that predict quantum evolution of systems with classical machine learning. Classical Shadow of quantum system. 2/3 Is there formal proof of quantum advantage. Sometimes yes, sometimes no. No for averages. 3/3 Yes for quantum information such as Pauli observables (spin on x,y,z axes) and represent positions after system transformation.
Chronopunk: A novel (Episode 3). Mongarthy leaves Stanford to Sydney. Doesn’t mix well with the elitist hubris in Palo Alto. Creativity, risk and inspiration versus accolades. What is science? (Feynman quote: “Paul Dirac had the courage to simply guess at the form of the equation and try to interpret it afterwards”. Guess, try, interpret. Mongarthy says: “This is science.” Building the Lisa character, excellence, sports, academics. Female macho. Can’t handle men if they don’t put her first. Pat dumps her. Mongarthy invites her to join his team and solve some difficult conceptional problems. in the time travel space. Mongarthy is descendant of Schrödinger, Feynman.
Universities grant system is a grift scheme and must be cut to save universities from themselves. Three indicators for why the current system is broken. 1/3 Tuition is shamefully high. 2/3 Ratio of admin to faculty is shamefully high 3/3 Productivity is low. Productivity as measured by how much useful research comes out per dollar invested. Universities should fund themselves through private money, tuition and services (like collaboration with companies). The US is not strong because of its universities but because of its entrepreneurs and the unique risk culture. Progress typically starts in the brain of a driven entrepreneur, not the academy. Masters degrees for foreign students are not about education but selling access to US system. Let the government make that money, not the universities.
Diffusion for text. Inception comes out of stealth mode. Diffusion for text is faster. Parallelism. More tokens/second, allows for faster iteration and better control. Less mistakes than autoregressive (rocket launch analogy). Solving key problem in diffusion for text is how to generate score functions for discrete data such as words? Relative to words structure. Score function looks at probability of words relative to each other. “For example, the sequential sampling of tokens is slow, hard to control, and often degrades”. “The xay sleeps” what is xay? Dog, Cat, pet etc…Somewhere else in the text there is “whiskers”, hence, cat is most likely.
Paper discussion. s1: Simple test-time scaling. Models learn how to solve problems. Curated dataset to train the model how to solve efficiently. Small, but high quality. Pre-train means gather knowledge and compress. Fine tune is find best path to extract knowledge. For that model must learn how to generate chain of thought process or traces. All traces are recycled into pre-training. Even bad traces are useful as data for pre-train. In particular RL works better, because the model remembers the bad path and switches in time. Test time scaling is when model applies flexible amount of compute in test time based on how well it’s doing. This can be done with problems that offer efficient evals. For example math or coding are good examples. Moravec paradox in reasoning. Interview with S1 author. Supervised fine tuning is like teaching a fisher men how to fish. RL is teach him how fish tastes so he figures out how to fish.
Shame to Geoffrey Hinton. Attacking Elon Musk with vulgar slander because he is fighting corruption! How low can a person sink? Geoffrey, if you must get political then: focus on 1/3 massive increase in tuition across all campuses, 2/3 rapid increase of ratio of admin to faculty across all campuses and 3/3 oppressive Woke virus on virtually all campuses. Instead you throw vulgar slander at the man who is inspiring a whole generation of kids to study physics and engineering in order to build. Just because he is touching your academic slush fund. Shame on you.
Three macro vectors driving markets. 1/3 DOGE. 2/3 Longterm bond yields coming down. 3/3 Shifts in global politics. DOGE purpose is to reduce waste and/or corruption with goal to balance budget. Removing corruption has multiplier effect on economy. Inverse of corruption. DOGE is not austerity. The opposite. Positive impact on real growth. Longterm bond yield coming down because Treasury not rolling financing to longterm (not reversing Yellen’s short-term funding). If US balances budget, 2 trillion USD less debt coming to market. Global politics. US shifting focus from Europe to China. China is only real competitor because their tech stacks up to US. Europe is not relevant. Proxy wars too expensive. Better invest in Asia theater. Defense spending must change. More cyber, AI and robotics.
Chronopunk: A novel (Episode 2)
We will drop episodes of the novel on a weekly basis. Thoughts about episode 2. Mody lands in a small Croatian beach in mid summer. He must find a way to make money. Can he use knowledge of the future? Like sports betting? No. But he can use skills, such as JiuJitsu? So he decides to become a prizefighter. Bogdan, a local manager, takes him under his wing. Mody succeeds and eventually makes it to Chicago. Thoughts: What does it mean to know about the future? Is there such a thing as “your” past and “your” future. What is a fact? What is knowledge. Facts depend on the trajectory of the multiverse, knowledge doesn’t?
Nvidia earnings call. Main drivers of Nvidia
business: 1/3 Reasoning 2/3 Agentic software development 3/3 Robotics. Jensen is the John Chambers of today. His earrings calls set the tone for where AI, technology and the economy are going. Software defined datacenter with CUDA at core. Nvidia metal and software allow for flexible reprogramming of datacenter. Training and task specific. Optimized to reduce token per dollar, token per MW. Risk to Nvidia is to the upside. With endeavors like π0 robotics will proliferate and increase infrastructure demand for training and inference. 50% growth. Cognitively capable agentic systems will literally create infinite wealth opportunities. Wealth is all possible transformation of a society.
Chronopunk Episode 1
Dropping episodes of novel on a weekly basis. Thoughts about episode 1. Science fiction. Only Ben Bernanke real. Setting the stage: Explaining how Mody gets to the be the first person to embark on an intergenerational journey through time from 2064 to 2006. His mission: Prevent the Fed from printing money. Main message: Future generations must have say when we waste their money. Quote by Rage Against The Machine: “Who controls the past now, controls the future. Who controls the present now, controls the past”. Essence of the novel. If you have the power to change the past, what does present and future even mean? Writing history is a serious job (see Google Gemini).Time travel physics? Different multiverse, but not yours (see Brett Hall).
Germany steers into oblivion by avoiding AFD. Election result is non-win for CDU. Voters want solutions but elites don’t have the courage, skill and guts to move towards leadership. Merkel era sleepwalking continues. Bad for Germany, bad for Europe. AFD is the only party which offers real solutions to real problems. Elitist blob doesn’t want to accept that. Germany must emancipate itself from post WWII guilt. JD Vance gave them an opening - “build a society based on Western values with freedom of speech at the core”. Germany is gliding into oblivion with post Covid, Woke infected moral high ground culture. “For Europe, the only thing worse than Germany as a leader is Germany as a non-leader”.
Chronopunk. (A novel)
“If you went back in time, what knowledge would you gift the past to save the future?”
Synopsis of novel
Reaction to Herbert Diess interview. Innovator’s Dilemma. My interpretation: when facing disruption, managerial class is buying time by going up market. New moral compass needed. Founder types don’t act like this. Tesla has been disrupting car market by offering software defined car with lower cost structure. Eventually, every company is defined by its cost structure. Now facing disruption in software defined car from Chinese and others. Response is full integration, lower cost, agility.
π0: A Vision-Language-Action Flow Model for General Robot Control. Sergey Levine et. al. Sergey talks about π0 model. Pre-train on VLA on internet scale data. Add robot specific actions for the Action part. Fine tune on specific tasks with experts. General means a lot of things in robotics. Very berkleyesqe = build large, distributed, open source systems (Cyberpunks, Spark, RiskV). Similar to Unix, which was designed as an operating system that is not hardware depended. π0 is robot hardware independent. Mix of experts. VLM + Action expert. Action is diffusion based. Pre-train data should be diverse, not just experts, so robot learns how to fix problems. Tokenization in robotics not optimized yet. Normalization of tokens. How much transfer learning is there from FSD to Optimus? Can there be a more general model? Robotics is in essence a navigation problem.
Tesla Q4 comments. 1/2 FSD competition in China. Best way to build real world AI system is with fully integrated system. From metal to algorithm to data collection. Tesla yes, Chinese no. Patchwork. Why is China not letting Tesla export training data? We let 400,000 Chinese students study in the US. Reciprocity please. 2/2 Battery constraint. Why? Demand or supply. Coordinating different supply chains.
Comments on JD Vance Munich speech and Lanz und Precht. JD Vance draws demarcation line between the US understanding of liberty and Europe. For centuries Europe has been building a culture of elitist liberty. The US starts from the bottom up, from the individual. Since 1945 the US has been trying to impose this idea of liberty on Europe. JD Vance now drew a line. No more funding and support. Europe must build their own sovereign system. That’s where I agree with Lanz und Precht. But Europe is not a political idea, it’s a geographic concept. Return to nation states. Align purse and sword with constituency.
Neurips 2024 Part 16.Return of Unconditional Generation. Kaiming He. The goal of representation learning is to compress input in latent space so it can be retrieved efficiently. Pixels are high dimensional. Model learns how to represent pixels. Feature extraction unsupervised. Presentation by Kaiming. Deep learning = representation learning. Input data is way too high dimensional. Necessity to reduce dimensionality into useful representation. History of deep learning: Recurring Neural Networks, Convolutional Neural Networks, Transformers. Same ideas, more cross attention. Normalization important in end to end learning.
DOGE, Tesla and macro. Doge impact will be like technology - “overestimated in the short run and massively underestimated in the long run.” Trump effect on Tesla. Europe yes, rest of world no. Tesla sales sluggish due to Juniper transition Model Y. DOGE upside. First time since invention of democracy in Athens that an empire is undying corruption. De-Byzantinizing through DOGE. If corruption slows down economy, the inverse accelerates it. We agree with ARK Invest that there is good chance for high single digit real growth in the coming five years. Tug of war between S&P500 and Gold. DOGE succeeds, S&P up, otherwise Gold.
Discussing Essay “From Shakespeare to Move 37”. Creativity evolves with the Zeitgeist. It depends on how we perceive learning. What is learning? What is the purpose of intelligence? Creativity is purposeful exploration. Today, artificial Intelligence opens new, interesting questions of what learning is and what creativity means. Move 37 is a first look at what modern AI system will be able to do. Knowledge and iteration through RL will evolve novel forms of creativity. The purpose of creativity is to build new niches and wealth.