# Spinning Up Deep Rl

> Navigate structured study notes derived from OpenAI’s Spinning Up documentation, with algorithm comparisons, experiment-design questions and links back to the original educational material.

- Canonical page: https://ai-skills-catalog.com/skills/spinning-up-deep-rl
- Category: AI/ML
- Author: Alireza Rezvani
- License: MIT
- Source: https://github.com/alirezarezvani/claude-skills/tree/main/engineering/spinning-up-deep-rl/skills/spinning-up-deep-rl
- Last updated: 2026-11-21
- Risk level: Low
- Declared permissions: Read project files; Write generated skill files

## Compatibility

- Claude: Packaged. Package: https://ai-skills-catalog.com/api/skills/spinning-up-deep-rl/package?agent=Claude
- Codex: Packaged. Package: https://ai-skills-catalog.com/api/skills/spinning-up-deep-rl/package?agent=Codex
- GPT: Packaged. Package: https://ai-skills-catalog.com/api/skills/spinning-up-deep-rl/package?agent=GPT
- Gemini: Packaged. Package: https://ai-skills-catalog.com/api/skills/spinning-up-deep-rl/package?agent=Gemini
- Cursor: Packaged. Package: https://ai-skills-catalog.com/api/skills/spinning-up-deep-rl/package?agent=Cursor
- OpenCode: Packaged. Package: https://ai-skills-catalog.com/api/skills/spinning-up-deep-rl/package?agent=OpenCode

## Sections of SKILL.md

- How to Use This Skill
- Core Frameworks & Mental Models
- The RL problem
- The two branching questions
- Policy optimization vs Q-learning — the central trade-off
- The policy gradient template
- The policy-gradient loss is not a loss function
- Broken RL code almost always fails silently
- Learn by doing
- Rigor: four standards
- The safe-step family (on-policy)
- The overestimation family (off-policy)
- Chapter Index
- Topic Index
- Supporting Files
- Scope & Limits

## Bundled files

- chapters/ch01-introduction.md
- chapters/ch02-installation.md
- chapters/ch03-algorithm-lineup.md
- chapters/ch04-running-experiments.md
- chapters/ch05-experiment-outputs.md
- chapters/ch06-plotting-results.md
- chapters/ch07-key-concepts-in-rl.md
- chapters/ch08-kinds-of-rl-algorithms.md
- chapters/ch09-intro-to-policy-optimization.md
- chapters/ch10-spinning-up-as-a-researcher.md
- chapters/ch11-key-papers-in-deep-rl.md
- chapters/ch12-exercises.md
- chapters/ch13-benchmarks.md
- chapters/ch14-vpg.md
- chapters/ch15-trpo.md
- chapters/ch16-ppo.md
- chapters/ch17-ddpg.md
- chapters/ch18-td3.md
- chapters/ch19-sac.md
- chapters/ch20-logger-and-utilities.md
- cheatsheet.md
- glossary.md
- patterns.md
- LICENSE

## Installation

### Claude

1. Download the Claude package from the skill page. The ZIP contains one folder named after the skill, with SKILL.md and any bundled reference files.
2. For Claude Code, unpack the folder into ~/.claude/skills/ to use it in every project, or into .claude/skills/ inside a repository to share it with your team.
3. For Claude apps, upload the ZIP in the Skills section of the capabilities settings.
4. Start a new session and describe the task in your own words. Claude picks the skill when the request matches its description.

### Codex

1. Download the Codex package. The ZIP contains one folder named after the skill, with SKILL.md and bundled resources.
2. Unpack the complete folder into .agents/skills/ inside your repository, or into ~/.agents/skills/ for personal use. Preserve its relative reference-file paths.
3. Start a new Codex session and invoke the skill by name or with $skill-name, or describe a matching task.
4. AGENTS.md is separate project guidance. The ZIP does not generate or modify that file.

### GPT

1. Download the GPT package from the skill page.
2. Create or edit a custom GPT (or a ChatGPT project) and paste the contents of SKILL.md into the instructions field.
3. Upload the bundled reference files as knowledge so the GPT can consult them.
4. Test with one of the example prompts from the skill page and adjust the instructions if your use case differs.

### Gemini

1. Download the Gemini package from the skill page.
2. For a project, unpack the complete folder into .gemini/skills/ or .agents/skills/. For personal use, use ~/.gemini/skills/ or ~/.agents/skills/.
3. Run /skills reload in Gemini CLI, then /skills list to check discovery. There is no need to paste the skill into GEMINI.md.
4. Test a matching task and verify the referenced tools and files are available.

### Cursor

1. Download the Cursor package from the skill page.
2. Unpack the complete skill folder into .cursor/skills/ or .agents/skills/ in your project. For personal local use, ~/.cursor/skills/ and ~/.agents/skills/ are also supported.
3. Start a new Cursor session. Type / in Agent chat and select the skill name, or describe a matching task.
4. Commit the skill folder and resources for team use. Personal local skills are not automatically available to remote agents or automation runs.

### OpenCode

1. Download the OpenCode package from the skill page.
2. Unpack the complete folder into .opencode/skills/ in your project, or ~/.config/opencode/skills/ for personal use. OpenCode also supports .agents/skills/ and .claude/skills/ compatibility locations.
3. Keep supporting files at their original relative paths. AGENTS.md is separate project-wide guidance and is not generated by this ZIP.
4. Start OpenCode in the project and test a matching task with the model and permissions you use.

## FAQ

### What does the Spinning Up Deep Rl skill do?

Navigate structured study notes derived from OpenAI’s Spinning Up documentation, with algorithm comparisons, experiment-design questions and links back to the original educational material.

### Which AI agents can use Spinning Up Deep Rl?

Packages for Claude, Codex and 4 more are built from the same source instructions.

### How do I install Spinning Up Deep Rl in Claude?

Download the Claude package from the skill page. The ZIP contains one folder named after the skill, with SKILL.md and any bundled reference files. For Claude Code, unpack the folder into ~/.claude/skills/ to use it in every project, or into .claude/skills/ inside a repository to share it with your team.

### Who made Spinning Up Deep Rl and what is the license?

Spinning Up Deep Rl is published by Alireza Rezvani under the MIT license. The source is https://github.com/alirezarezvani/claude-skills/tree/main/engineering/spinning-up-deep-rl/skills/spinning-up-deep-rl. The catalog keeps the original attribution.

### What permissions does Spinning Up Deep Rl need?

Declared permissions: read project files, write generated skill files. Risk level: low. Check the scripts in the source before running them.

## Related AI/ML skills

- [Chief AI Officer Advisor](https://ai-skills-catalog.com/skills/chief-ai-officer-advisor-2): Evaluate AI strategy through model sourcing, risk governance, workload economics and organization design.
- [Senior ML Engineer](https://ai-skills-catalog.com/skills/senior-ml-engineer): Plan production model delivery, MLOps pipelines, LLM integration and retrieval-augmented generation around deployment and monitoring needs.
- [Chief AI Officer Advisor](https://ai-skills-catalog.com/skills/chief-ai-officer-advisor): Compare strategic AI investment and governance options.
- [Prompt Engineer Toolkit](https://ai-skills-catalog.com/skills/prompt-engineer-toolkit): Treat prompts as versioned assets with test cases, comparisons and promotion evidence.
- [Senior Computer Vision](https://ai-skills-catalog.com/skills/senior-computer-vision): Prepare vision datasets, training configurations and inference investigations around a defined detection or segmentation task, with deployment limits and evaluation requirements made explicit.
- [Deep Learning Book](https://ai-skills-catalog.com/skills/deep-learning-book): Use an original study companion to navigate Deep Learning, plan prerequisite reading and examine training diagnostics or model arithmetic without replacing the source textbook.
