Storybook: Include playground stories and MDX in CI smoke tests#79454
Conversation
|
Size Change: 0 B Total Size: 7.5 MB |
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
Flaky tests detected in b853fe2. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/28044830276
|
aduth
left a comment
There was a problem hiding this comment.
Thanks for the context in tracking down the history here. At least as far as my own recent contributions are concerned, as mentioned it was largely carrying forward what had already existed, and adding documentation to clarify what my own interpretation of the exclusions would have been intended for, without critically considering whether they're valuable. If they're not contributing meaningfully to build times and this has proven to have caused errors on trunk, it seems reasonable (and simpler overall) to remove the exclusion.
Also good to clarify about the impact of potentially using 'production' here, to "fully" recreate a trunk build. Given that has a more noticeable impact on build times and the minification itself doesn't seem quite as prone to issues, seems reasonable to keep NODE_ENV: test for now.
What?
Stop excluding playground stories and top-level Storybook MDX files from the CI smoke test build (
NODE_ENV=test).Why?
The smoke test workflow (
storybook-check.yml) runs withNODE_ENV=test, while the GitHub Pages deploy usesNODE_ENV=production. The conditionals instorybook/main.tsmeant CI only built a subset of stories, so production-only build failures could slip through. This gap recently caused a merged PR to pass CI while breaking the Storybook Pages deploy.Benchmarking locally shows the exclusions save only ~1 second of storybook build time (~1.5%), so the coverage trade-off is likely not worthwhile, even on CI.
The
NODE_ENV === 'test'story exclusions date back to the Storybook 5.3 era (#19599), where they were used for Storyshots. They were carried forward through later Storybook upgrades (#53520, #74396) without a documented rationale tied to current CI.#69679 added the smoke test workflow but did not introduce these conditionals or set
NODE_ENVin CI. Until #74626 wired upNODE_ENV: testinstorybook-check.yml, the exclusions were likely not even taking effect in CI. #74626 narrowed them from all./stories/**files to playground stories and top-level MDX, and added the comment about speed and redundancy during review. That PR was primarily about enabling the components manifest.How?
Always include
./stories/playground/**/*.story.@(jsx|tsx)and./stories/**/*.mdxin the stories glob list.Note
You may be asking, should we stop doing
NODE_ENV=testfor the CI run then? My conclusion is no, because the main difference remaining betweentestandproductionis minification, and that indeed does contribute a non-negligible amount of time (~8s).Testing Instructions
NODE_ENV=test npm run storybook:buildbefore and after.