8000 Fold Travis CI output by component by nicolas-grekas · Pull Request #22461 · symfony/symfony · GitHub
[go: up one dir, main page]
Skip to content

Fold Travis CI output by component #22461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 19, 2017
Merged

Fold Travis CI output by component #22461

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7a9b086
Fold Travis CI output by component
maidmaid Apr 3, 2017
cf87678
Make .travis.yml more readable
nicolas-grekas Apr 18, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 119 additions & 47 deletions .travis.yml
View file
Open in desktop
8000
Original file line number Diff line number Diff line change
Expand Up @@ -40,55 +40,127 @@ cache:
services: mongodb

before_install:
- stty cols 120
- PHP=$TRAVIS_PHP_VERSION
# Matrix lines for intermediate PHP versions are skipped for pull requests
- if [[ ! $deps && ! $PHP = ${MIN_PHP%.*} && ! $PHP = hhvm* && $TRAVIS_PULL_REQUEST != false ]]; then deps=skip; skip=1; fi
# A sigchild-enabled-PHP is used to test the Process component on the lowest PHP matrix line
- if [[ ! $deps && $PHP = ${MIN_PHP%.*} && ! -d php-$MIN_PHP/sapi ]]; then wget http://museum.php.net/php5/php-$MIN_PHP.tar.bz2 -O - | tar -xj; (cd php-$MIN_PHP; ./configure --enable-sigchild --enable-pcntl; make -j2); fi
- if [[ ! $PHP = hhvm* ]]; then INI_FILE=~/.phpenv/versions/$(phpenv version-name)/etc/conf.d/travis.ini; else INI_FILE=/etc/hhvm/php.ini; fi
- if [[ ! $skip ]]; then echo date.timezone = Europe/Paris >> $INI_FILE; fi
- if [[ ! $skip ]]; then echo memory_limit = -1 >> $INI_FILE; fi
- if [[ ! $skip ]]; then echo session.gc_probability = 0 >> $INI_FILE; fi
- if [[ ! $skip ]]; then echo opcache.enable_cli = 1 >> $INI_FILE; fi
- if [[ ! $skip ]]; then echo hhvm.jit = 0 >> $INI_FILE; fi
- if [[ ! $skip && $PHP = 5.* ]]; then echo extension = mongo.so >> $INI_FILE; fi
- if [[ ! $skip && $PHP = 5.* ]]; then echo extension = memcache.so >> $INI_FILE; fi
- if [[ ! $skip && $PHP = 5.* ]]; then (echo yes | pecl install -f apcu-4.0.11 && echo apc.enable_cli = 1 >> $INI_FILE); fi
- if [[ ! $skip && $PHP = 7.* ]]; then (echo yes | pecl install -f apcu-5.1.6 && echo apc.enable_cli = 1 >> $INI_FILE); fi
- if [[ ! $deps && $PHP = 5.* ]]; then (cd src/Symfony/Component/Debug/Resources/ext && phpize && ./configure && make && echo extension = $(pwd)/modules/symfony_debug.so >> $INI_FILE); fi
- if [[ ! $skip && $PHP = 5.* ]]; then pecl install -f memcached-2.1.0; fi
- if [[ ! $skip && ! $PHP = hhvm* ]]; then echo extension = ldap.so >> $INI_FILE; fi
- if [[ ! $skip && ! $PHP = hhvm* ]]; then phpenv config-rm xdebug.ini || echo "xdebug not available"; fi
- if [[ ! $skip ]]; then [ -d ~/.composer ] || mkdir ~/.composer; cp .composer/* ~/.composer/; fi
- if [[ ! $skip ]]; then export PHPUNIT=$(readlink -f ./phpunit); fi
- |
# General configuration
stty cols 120
PHP=$TRAVIS_PHP_VERSION
[ -d ~/.composer ] || mkdir ~/.composer
cp .composer/* ~/.composer/
export PHPUNIT=$(readlink -f ./phpunit)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you sure that failures in these commands actually make Travis mark the build as errored ? the main Travis build script does not run with -e (it would break their own reporting). I think they would report an error only for the last statement of your big string

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set -e now moved at the beginning of the file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://travis-ci.org/symfony/symfony/jobs/223219302

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a bad idea, as it will put the whole Travis script in -e mode, breaking Travis features (see that your build output does not have the Travis failure report at the end). This is also why the Travis docs says we should never use exit in a script command.

If you want to use -e, you have to make it in a script executed through sh, not directly in the main commands.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed, fixed by chaining commands with &&

Copy link
Member
@stof stof Apr 19, 2017 •
edited by nicolas-grekas
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this is incomplete

Copy link
Member Author
@nicolas-grekas nicolas-grekas Apr 19, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need this for all commands, the missing ones are on purpose

export PHPUNIT_X="$PHPUNIT --exclude-group tty,benchmark,intl-data"
export COMPOSER_UP='composer update --no-progress --no-suggest --ansi'

# tfold is a helper to create folded reports
tfold () {
title=$1
fold=$(echo $title | sed -r 's/[^-_A-Za-z\d]+/./g')
shift
echo -e "travis_fold:start:$fold\\n\\e[1;34m$title\\e[0m"
bash -xc "$*" 2>&1 &&
echo -e "\\e[32mOK\\e[0m $title\\n\\ntravis_fold:end:$fold" ||
( echo -e "\\e[41mKO\\e[0m $title\\n" && exit 1 )
}
export -f tfold

# php.ini configuration
if [[ $PHP = hhvm* ]]; then
INI=/etc/hhvm/php.ini
else
INI=~/.phpenv/versions/$(phpenv version-name)/etc/conf.d/travis.ini
phpenv config-rm xdebug.ini || echo "xdebug not available"
fi
echo date.timezone = Europe/Paris >> $INI
echo memory_limit = -1 >> $INI
echo session.gc_probability = 0 >> $INI
echo opcache.enable_cli = 1 >> $INI
echo hhvm.jit = 0 >> $INI
echo apc.enable_cli = 1 >> $INI
echo extension = ldap.so >> $INI
[[ $PHP = 5.* ]] && echo extension = mongo.so >> $INI
[[ $PHP = 5.* ]] && echo extension = memcache.so >> $INI

# Matrix lines for intermediate PHP versions are skipped for pull requests
if [[ ! $deps && ! $PHP = ${MIN_PHP%.*} && ! $PHP = hhvm* && $TRAVIS_PULL_REQUEST != false ]]; then
deps=skip
skip=1
else
COMPONENTS=$(find src/Symfony -mindepth 3 -type f -name phpunit.xml.dist -printf '%h\n')
fi

- |
# Install sigchild-enabled PHP to test the Process component on the lowest PHP matrix line
if [[ ! $deps && $PHP = ${MIN_PHP%.*} && ! -d php-$MIN_PHP/sapi ]]; then
wget http://museum.php.net/php5/php-$MIN_PHP.tar.bz2 -O - | tar -xj &&
(cd php-$MIN_PHP && ./configure --enable-sigchild --enable-pcntl && make -j2)
fi

- |
# Install extra PHP extensions
if [[ ! $skip && $PHP = 5.* ]]; then
([[ $deps ]] || tfold ext.symfony_debug 'cd src/Symfony/Component/Debug/Resources/ext && phpize && ./configure && make && echo extension = $(pwd)/modules/symfony_debug.so >> '"$INI") &&
tfold ext.memcached pecl install -f memcached-2.1.0 &&
tfold ext.apcu4 'echo yes | pecl install -f apcu-4.0.11'
elif [[ ! $skip && $PHP = 7.* ]]; then
tfold ext.apcu5 'echo yes | pecl install -f apcu-5.1.6'
fi

install:
- if [[ ! $skip && $deps ]]; then cp composer.json composer.json.orig; fi
- if [[ ! $skip && $deps ]]; then echo -e '{\n"require":{'"$(grep phpunit-bridge composer.json)"'"php":"*"},"minimum-stability":"dev"}' > composer.json; fi
- if [[ ! $skip ]]; then COMPONENTS=$(find src/Symfony -mindepth 3 -type f -name phpunit.xml.dist -printf '%h\n'); fi
# Create local composer packages for each patched components and reference them in composer.json files when cross-testing components
- if [[ ! $skip && $deps ]]; then php .github/build-packages.php HEAD^ $COMPONENTS; fi
- if [[ ! $skip && $deps ]]; then mv composer.json composer.json.phpunit; mv composer.json.orig composer.json; fi
- if [[ ! $skip && ! $deps ]]; then php .github/build-packages.php HEAD^ src/Symfony/Bridge/PhpUnit; fi
# For the master branch when deps=high, the version before master is checked out and tested with the locally patched components
- if [[ $deps = high && $TRAVIS_BRANCH = master ]]; then SYMFONY_VERSION=$(git ls-remote --heads | grep -o '/[1-9].*' | tail -n 1 | sed s/.//); else SYMFONY_VERSION=$(cat composer.json | grep '^ *"dev-master". *"[1-9]' | grep -o '[0-9.]*'); fi
- if [[ $deps = high && $TRAVIS_BRANCH = master ]]; then git fetch origin $SYMFONY_VERSION; git checkout -m FETCH_HEAD; COMPONENTS=$(find src/Symfony -mindepth 3 -type f -name phpunit.xml.dist -printf '%h\n'); fi
# Legacy tests are skipped when deps=high and when the current branch version has not the same major version number than the next one
- if [[ $deps = high && ${SYMFONY_VERSION%.*} != $(git show $(git ls-remote --heads | grep -FA1 /$SYMFONY_VERSION | tail -n 1):composer.json | grep '^ *"dev-master". *"[1-9]' | grep -o '[0-9]*' | head -n 1) ]]; then LEGACY=,legacy; fi
- export COMPOSER_ROOT_VERSION=$SYMFONY_VERSION.x-dev
- if [[ ! $skip && $deps ]]; then export SYMFONY_DEPRECATIONS_HELPER=weak; fi
- if [[ ! $skip && $deps ]]; then mv composer.json.phpunit composer.json; fi
- if [[ ! $skip ]]; then composer update --no-suggest; fi
- |
# Create local composer packages for each patched components and reference them in composer.json files when cross-testing components
if [[ ! $deps ]]; then
php .github/build-packages.php HEAD^ src/Symfony/Bridge/PhpUnit
elif [[ ! $skip ]]; then
export SYMFONY_DEPRECATIONS_HELPER=weak &&
cp composer.json composer.json.orig &&
echo -e '{\n"require":{'"$(grep phpunit-bridge composer.json)"'"php":"*"},"minimum-stability":"dev"}' > composer.json &&
php .github/build-packages.php HEAD^ $COMPONENTS &&
mv composer.json composer.json.phpunit &&
mv composer.json.orig composer.json
fi

- |
# For the master branch, when deps=high, the version before master is checked out and tested with the locally patched components
if [[ $deps = high && $TRAVIS_BRANCH = master ]]; then
SYMFONY_VERSION=$(git ls-remote --heads | grep -o '/[1-9].*' | tail -n 1 | sed s/.//) &&
git fetch origin $SYMFONY_VERSION &&
git checkout -m FETCH_HEAD &&
COMPONENTS=$(find src/Symfony -mindepth 3 -type f -name phpunit.xml.dist -printf '%h\n')
elif [[ ! $skip ]]; then
SYMFONY_VERSION=$(cat composer.json | grep '^ *"dev-master". *"[1-9]' | grep -o '[0-9.]*')
fi

- |
# Legacy tests are skipped when deps=high and when the current branch version has not the same major version number than the next one
[[ $deps = high && ${SYMFONY_VERSION%.*} != $(git show $(git ls-remote --heads | grep -FA1 /$SYMFONY_VERSION | tail -n 1):composer.json | grep '^ *"dev-master". *"[1-9]' | grep -o '[0-9]*' | head -n 1) ]] && LEGACY=,legacy

export COMPOSER_ROOT_VERSION=$SYMFONY_VERSION.x-dev
if [[ ! $skip && $deps ]]; then mv composer.json.phpunit composer.json; fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we actually need this one ? our PHPUnit wrapper already handles the installation of the bridge, not need to force again the usage of the one requiring it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this allows using the latest version of the bridge (maybe patched by the current PR), with a previous version of the code base (eg on deps=high on master branch where we checkout 3.2)
so yes we need it


- if [[ ! $skip ]]; then $COMPOSER_UP; fi
- if [[ ! $skip ]]; then ./phpunit install; fi
- if [[ ! $skip && ! $PHP = hhvm* ]]; then php -i; else hhvm --php -r 'print_r($_SERVER);print_r(ini_get_all());'; fi
- |
# phpinfo
if [[ ! $PHP = hhvm* ]]; then php -i; else hhvm --php -r 'print_r($_SERVER);print_r(ini_get_all());'; fi

- |
run_tests () {
set -e
if [[ $skip ]]; then
echo -e "\\n\\e[1;34mIntermediate PHP version $PHP is skipped for pull requests.\\e[0m"
elif [[ $deps = high ]]; then
echo "$COMPONENTS" | parallel --gnu -j10% "tfold {} 'cd {} && $COMPOSER_UP && $PHPUNIT_X$LEGACY'"
elif [[ $deps = low ]]; then
echo "$COMPONENTS" | parallel --gnu -j10% "tfold {} 'cd {} && $COMPOSER_UP --prefer-lowest --prefer-stable && $PHPUNIT_X'"
elif [[ $PHP = hhvm* ]]; then
$PHPUNIT --exclude-group benchmark,intl-data
else
echo "$COMPONENTS" | parallel --gnu "tfold {} $PHPUNIT_X {}"
tfold tty-group $PHPUNIT --group tty
if [[ $PHP = ${MIN_PHP%.*} ]]; then
echo -e "1\\n0" | xargs -I{} bash -c "tfold src/Symfony/Component/Process.sigchild{} ENHANCE_SIGCHLD={} php-$MIN_PHP/sapi/cli/php .phpunit/phpunit-4.8/phpunit --colors=always src/Symfony/Component/Process/"
fi
fi
}

script:
- REPORT=' && echo -e "\\e[32mOK\\e[0m {}\\n\\n" || (echo -e "\\e[41mKO\\e[0m {}\\n\\n" && $(exit 1))'
- if [[ $skip ]]; then echo -e "\\n\\e[1;34mIntermediate PHP version $PHP is skipped for pull requests.\\e[0m"; fi
- if [[ ! $deps && ! $PHP = hhvm* ]]; then echo "$COMPONENTS" | parallel --gnu '$PHPUNIT --exclude-group tty,benchmark,intl-data {}'"$REPORT"; fi
- if [[ ! $deps && ! $PHP = hhvm* ]]; then echo -e "\\nRunning tests requiring tty"; $PHPUNIT --group tty; fi
- if [[ ! $deps && $PHP = hhvm* ]]; then $PHPUNIT --exclude-group benchmark,intl-data; fi
- if [[ ! $deps && $PHP = ${MIN_PHP%.*} ]]; then echo -e "1\\n0" | xargs -I{} sh -c 'echo "\\nPHP --enable-sigchild enhanced={}" && ENHANCE_SIGCHLD={} php-$MIN_PHP/sapi/cli/php .phpunit/phpunit-4.8/phpunit --colors=always src/Symfony/Component/Process/'; fi
- if [[ $deps = high ]]; then echo "$COMPONENTS" | parallel --gnu -j10% 'cd {}; composer update --no-progress --no-suggest --ansi; $PHPUNIT --exclude-group tty,benchmark,intl-data'$LEGACY"$REPORT"; fi
- if [[ $deps = low ]]; then echo "$COMPONENTS" | parallel --gnu -j10% 'cd {}; composer update --no-progress --no-suggest --ansi --prefer-lowest --prefer-stable; $PHPUNIT --exclude-group tty,benchmark,intl-data'"$REPORT"; fi
- (run_tests)
0