Tailwindui Crawler

tailwindui-crawler downloads the component HTML files locally

Download

tailwindui-crawler

This script will crawl the tailwindcss.com/plus website and download all the components to the ./output folder.

🛠 How to use

To install, clone this repo and run yarn or npm install to pull down the dependencies.

Then create a .env file with your email, password, and optional output folder.

EMAIL=youremail
PASSWORD=yourpassword
# OUTPUT optional, defaults to ./output
OUTPUT=/path/to/output
# LANGUAGES defaults to html
LANGUAGES=html,react,vue,alpine
# COMPONENTS commma-delimited list (defaults to all)
COMPONENTS=(all|marketing,application-ui,ecommerce)
# BUILDINDEX generate index file to view components offline
BUILDINDEX=(0 | 1)
# TEMPLATES download template files
TEMPLATES=(0 | 1)
# FORCE_UPDATE force update existing components
FORCE_UPDATE=(0 | 1)
# DEBUG enable debug logging
DEBUG=(0 | 1)

NOTE: The tool uses dotenv-expand to support variable expansion like $HOME/path/to/output so if your password or any other value includes a $, make sure you add a \ (backslash) to escape the $. For example, PASSWORD=p@\$\$w0rd

Also, dotenv does not support inline comments, so do not do something like LANGUAGES=html,react,vue # some comment as it will not get the correct values.

Then finally, run yarn start or npm start

The script will login to tailwindcss.com/plus with your credentials, and download all the components as individual files in the ./output folder.

🤔 What's it for?

The benefit of pulling down all the components is that you can commit them to a local or private repo, and by running this periodically, you can see exactly which files were added or changed. Hopefully, some time in the future, they will open up a private repo for those that have purchased the library.

🚀 New v5.0

The crawler has been updated to support the new Tailwind Plus site structure (tailwindcss.com/plus) as of 2024. It includes support for downloading all marketing UI blocks and properly handles the new component structure.

New Features:

Support for the new Tailwind Plus URL structure
Enhanced component naming for React components (generates descriptive names based on the component path)
Added FORCE_UPDATE option to overwrite existing components
Improved debugging with the DEBUG environment variable
Component skipping to avoid re-downloading existing components
Detailed summary statistics after download completes

🗂 Preview page

You can set the .env key BUILDINDEX=1 to have the crawler generate an index file similar to the components page on tailwindcss.com/plus. Install and run the serve package to view the index.

NOTE: The HTML Preview does not apply transformers. It's a copy of the component site on tailwindcss.com/plus.

yarn global add serve
cd $OUTPUT/preview # change to your preview folder
serve

⚙️ Example `.env` file

EMAIL=******
PASSWORD=******
OUTPUT=$HOME/Projects/tailwindui
LANGUAGES=html,react,vue,alpine
COMPONENTS=marketing,application-ui
BUILDINDEX=1
TEMPLATES=1
FORCE_UPDATE=0
DEBUG=0

🤖 Automatically keep a private GitHub Repository up-to-date

NOTE: GitHub action has been updated in v4.0.0. Please make sure your default.yml file is updated with the latest actions.

You can automatically keep a private GitHub repository up-to-date with component changes from TailwindUI by using this tool with GitHub Actions.

Create a private GitHub repository.
Add TAILWINDUI_EMAIL and TAILWINDUI_PASSWORD secrets to the GitHub repository.
Optionally create a .env file with additional settings for the crawler.

Create a new file .github/workflows/default.yml:

name: Update
on:
  schedule:
    - cron: '0 0 * * *' # Every day at midnight

jobs:
  update:
    name: Update
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Run crawler
        uses: kiliman/[email protected]
        with:
          email: ${{ secrets.TAILWINDUI_EMAIL }}
          password: ${{ secrets.TAILWINDUI_PASSWORD }}

NOTE: Make sure to update to the latest action v1.1.0 to support the crawler v3+

Read more about the schedule cron syntax in the official GitHub Actions documentation.

NOTE: if you're creating a new repository or have updated your default branch from master, you will have to specify with branch (and/or current_branch) like so:

# ...
      - name: Run crawler
        uses: kiliman/[email protected]
        with:
          email: ${{ secrets.TAILWINDUI_EMAIL }}
          password: ${{ secrets.TAILWINDUI_PASSWORD }}
+         branch: main
+         current_branch: main

Email Notifications

To be emailed whenever there is a change to a component, simply setup GitHub Notifications on your repository.

🚦 Upgrading to v5.

This is a major change that supports the new Tailwind Plus site structure. The crawler now downloads components from tailwindcss.com/plus instead of the old tailwindui.com site.

New features:

Better React component naming based on component paths
Skipping existing components for faster updates
Option to force update with FORCE_UPDATE=1
Enhanced debug output with DEBUG=1
Support for marketing components and all UI blocks

NOTE: Since this script is essentially screen scraping, there's the potential of it breaking if the HTML structure changes. I will do my best to keep it in sync with the website.

😍 Thank you

Thanks to Adam and Steve for making an amazing library. This has definitely made creating a UI for my applications a pleasant experience.

Enjoy and let me know if you have any questions.

Kiliman

Contributors ✨

Thanks goes to these wonderful people (emoji key):

_Kiliman 💻	_{Simon Waloschek} 💻	_{Pavel Fomchenkov} 💻	_{Robin Malfait} 💻	_{Miguel Piedrafita} 💻 📖 🤔	_{Vlad Dumitrescu} 📖	_C-Bass 💻
_{Greg Brimble} 📖 🔧	_Yagnik 💻	_idebeijer 💻