Solving issues with CI/CD Pipelines and Composer 2.7.0

In Deploying to Acquia Cloud with Bitbucket Pipelines, we discussed setting up Bitbucket Pipelines to construct a CI/CD environment that runs composer install and gulp to build the code and then pushes the built code to our client's Acquia Cloud instance.

We recently uncovered an issue with our pipelines caused by changes in Composer 2.7.0 that caused our pipeline process to push incorrectly built code.

TL;DR

If your code is not building correctly and your Drupal dependencies are installing into vendor/drupal/, check the version of Composer being used to build your code. If it is >=2.7.0, add export COMPOSER_ALLOW_SUPERUSER=1 to your build script directly before your composer commands.

Background

Composer-based Drupal sites rely on Composer plugins such as composer/installers and drupal/core-composer-scaffold to ensure that dependencies get installed into a folder structure necessary for the Drupal site to run, usually in docroot/ or web/ depending on your desired web root.

Specifying the web root in the project's composer.json file tells drupal/core-composer-scaffold where in the folder structure to place files that are included as part of the Drupal core package.

Including the composer/installers plugin in your composer.json file and populating the installer-paths section ensures that Drupal core, contributed modules and themes get built in the correct place in the folder structure.

The issue

If these two plugins are not in use, Drupal core and any contributed modules, libraries or themes get installed into the vendor/drupal/ folder in the repository. When that code gets pushed, Acquia Cloud doesn't see a valid site installed as the necessary files and folders are not where it expects to find them. Instead we get the Install Drupal screen.

Changes to Composer in response to a security vulnerability meant that from version 2.7.0, if you were running composer install "unattended" as a super-user, e.g. in a Bitbucket or GitLab pipeline, then Composer plugins are disabled by default.

The fix

Composer 2.7.1 was quickly released after 2.7.0, adding a warning about this and directing you how to fix the issue. The following warning is output:

Composer plugins have been disabled for safety in this non-interactive session. Set COMPOSER_ALLOW_SUPERUSER=1 if you want to allow plugins to run as root/superuser.

(See the Github issue #11839 Composer 2.7.0 usage with root user leads to plugins not being loaded for more information.)

In our case, we needed to update our build.sh file to set the  COMPOSER_ALLOW_SUPERUSER environment variable to  1.

Once we had done this, Composer used the correct plugins and built the project code correctly, which was then pushed to Acquia Cloud, and the site operated as expected.

A note on caching

We didn't experience this issue on all git branches on projects, just new ones. This was because the Bitbucket and Gitlab CI/CD environments associated with existing branches were still using cached versions of Composer.