2017-02-24 09:13:04 -05:00
|
|
|
#!/bin/sh
|
|
|
|
export LC_ALL=C
|
|
|
|
LABEL=$1
|
2017-07-24 14:44:12 +02:00
|
|
|
# Remove the label from arguments. It gets special treatment. Other arguments
|
|
|
|
# to the script are passed to pungi-koji directly.
|
|
|
|
shift
|
2017-02-24 09:13:04 -05:00
|
|
|
CONFIG="fedora-atomic.conf"
|
|
|
|
TARGET_DIR="/mnt/koji/compose/twoweek"
|
|
|
|
#OLD_COMPOSES_DIR="--old-composes=/mnt/fedora_koji/compose/f23 --old-composes=$TARGET_DIR"
|
|
|
|
NIGHTLY=""
|
|
|
|
SKIP_PHASES="--skip-phase=productimg --skip-phase=extra_files"
|
|
|
|
DEST=$(pwd)
|
|
|
|
DATE=$(date "+%Y%m%d")
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
# the Pungi 'shortname', which we will include in fedmsgs for disambiguation
|
|
|
|
SHORT="Fedora-Atomic"
|
|
|
|
RELEASE="27"
|
|
|
|
RELEASE_TITLE="27"
|
2017-10-22 21:39:01 -04:00
|
|
|
COMPSFILE="comps-f27.xml"
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
TMPDIR=`mktemp -d /tmp/$RELEASE.$DATE.XXXX`
|
2017-02-24 09:13:04 -05:00
|
|
|
TOMAIL="devel@lists.fedoraproject.org test@lists.fedoraproject.org"
|
|
|
|
FROM="Fedora Branched Report <rawhide@fedoraproject.org>"
|
|
|
|
RSYNCPREFIX="sudo -u ftpsync"
|
2018-01-25 11:00:52 +01:00
|
|
|
RSYNCTARGET="/pub/alt/atomic/testing/$SHORTCOMPOSE_ID"
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
OLDCOMPOSE_ID=$(cat $TARGET_DIR/latest-$SHORT-$RELEASE_TITLE/COMPOSE_ID)
|
2017-02-24 09:13:04 -05:00
|
|
|
# uncomment and edit for resuming a failed compose
|
|
|
|
#COMPOSE_ID="Fedora-23-20150530.n.0"
|
|
|
|
# assume a releng dir is a git checkout of teh releng repo
|
|
|
|
# if it does not exist clone it
|
|
|
|
if [ -d releng ]; then
|
|
|
|
pushd releng
|
|
|
|
git pull --rebase
|
|
|
|
popd
|
|
|
|
else
|
|
|
|
git clone https://pagure.io/releng.git
|
|
|
|
fi
|
|
|
|
# Set up our fedmsg function, using the releng repo definition
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
fedmsg_json_start=$(printf '{"log": "start", "branch": "%s", "arch": "%s", "short": "%s"}' "$RELEASE" "$ARCH" "$SHORT")
|
2017-02-24 09:13:04 -05:00
|
|
|
FEDMSG_MODNAME="compose"
|
|
|
|
FEDMSG_CERTPREFIX="releng"
|
|
|
|
. ./releng/scripts/fedmsg-functions.sh
|
|
|
|
# Announce that we are starting...
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
send_fedmsg "${fedmsg_json_start}" ${RELEASE} start
|
2017-02-24 09:13:04 -05:00
|
|
|
pushd $TMPDIR
|
|
|
|
git clone https://pagure.io/fedora-comps.git && {
|
|
|
|
pushd fedora-comps
|
|
|
|
make "${COMPSFILE}"
|
|
|
|
cp "${COMPSFILE}" $DEST/
|
|
|
|
popd
|
|
|
|
}
|
|
|
|
popd
|
|
|
|
CMD="pungi-koji --notification-script=/usr/bin/pungi-fedmsg-notification --config=$CONFIG --old-composes=$TARGET_DIR $OLD_COMPOSES_DIR $NIGHTLY $SKIP_PHASES --label=$LABEL"
|
|
|
|
if [ -z "$COMPOSE_ID" ]; then
|
|
|
|
CMD="$CMD --target-dir=$TARGET_DIR"
|
|
|
|
else
|
|
|
|
CMD="$CMD --debug-mode --compose-dir=$TARGET_DIR/$COMPOSE_ID"
|
|
|
|
fi
|
|
|
|
time $CMD "$@"
|
2017-12-04 15:44:52 -08:00
|
|
|
if [ "$?" != "0" ]; then
|
|
|
|
exit 1
|
2017-02-24 09:13:04 -05:00
|
|
|
fi
|
2017-12-04 15:44:52 -08:00
|
|
|
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
NEWCOMPOSE_ID=$(cat $TARGET_DIR/latest-$SHORT-$RELEASE_TITLE/COMPOSE_ID)
|
2017-02-24 09:13:04 -05:00
|
|
|
SHORTCOMPOSE_ID=$(echo $NEWCOMPOSE_ID|sed -e 's|Fedora-.*-||g')
|
2017-12-04 15:44:52 -08:00
|
|
|
|
2017-02-24 09:13:04 -05:00
|
|
|
DESTDIR=$TARGET_DIR/$NEWCOMPOSE_ID
|
2018-01-25 11:00:52 +01:00
|
|
|
# Public URL the synced compose will wind up at, we put it in fedmsgs
|
2018-01-25 12:28:54 +01:00
|
|
|
LOCATION="https://dl.fedoraproject.org$RSYNCTARGET"
|
2018-01-25 11:00:52 +01:00
|
|
|
# Update fedmsg template
|
|
|
|
fedmsg_json_start=$(printf '{"log": "start", "branch": "%s", "arch": "%s", "short": "%s", "compose_id": "%s", "location": "%s"}' "$RELEASE" "$ARCH" "$SHORT" "$NEWCOMPOSE_ID", "$LOCATION")
|
|
|
|
fedmsg_json_done=$(printf '{"log": "done", "branch": "%s", "arch": "%s", "short": "%s", "compose_id": "%s", "location": "%s"}' "$RELEASE" "$ARCH" "$SHORT" "$NEWCOMPOSE_ID" "$LOCATION")
|
|
|
|
|
2017-02-24 09:13:04 -05:00
|
|
|
$RSYNCPREFIX mkdir -p $DESTDIR
|
|
|
|
# Tell interested persons that the rsync is starting (zomg!)
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
send_fedmsg "${fedmsg_json_start}" ${RELEASE} rsync.start
|
2017-02-24 09:13:04 -05:00
|
|
|
for dir in Atomic CloudImages Docker metadata ;
|
|
|
|
do
|
2018-01-25 12:12:21 +01:00
|
|
|
$RSYNCPREFIX rsync -avhH $DESTDIR/compose/$dir/ "$RSYNCTARGET/$dir/" ;
|
2018-01-25 11:00:52 +01:00
|
|
|
$RSYNCPREFIX ./releng/scripts/build_composeinfo "$RSYNCTARGET/" --name $NEWCOMPOSE_ID
|
2017-02-24 09:13:04 -05:00
|
|
|
done
|
|
|
|
# Tell interested persons that the rsync is done.
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
send_fedmsg "${fedmsg_json_done}" ${RELEASE} rsync.complete
|
2017-02-24 09:13:04 -05:00
|
|
|
# Tell everyone by fedmsg about the compose
|
Add 'short' key to post-release compose fedmsgs
The various post-release composes - two-week Atomic, Cloud, and
Docker - all send out identical fedmsgs. The other commits in
this PR at least ensure the fedmsgs sent after the compose is
done will have a `compose_id` key, allowing disambiguation that
way, but fedmsgs sent *before* the compose is done cannot be
told apart.
This should solve that, without inventing new message topics:
we add a `short` key to the message content which specifies the
'shortname' of the compose (e.g. Fedora-Atomic), which is how
we distinguish these composes from each other.
We also remove a bunch of unused variables from these files,
and revise the release-related variables as explained for the
master branch in #485: `$DIST`, `$BRANCH` and `$BRANCHED` are
all replaced by `$RELEASE` and `$RELEASE_TITLE`.
This does duplicate the value from the pungi config file, but
I'm not sure it's worth figuring out a bash-y way to parse the
value out of the config file just to remove this duplication.
And we already effectively did this when discovering
`OLDCOMPOSE_ID` and `NEWCOMPOSE_ID`, as the path used there is
dependent upon the shortname; to avoid extra duplication, I've
changed those to use `$SHORT-$RELEASE_TITLE`.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 16:03:52 -08:00
|
|
|
send_fedmsg "${fedmsg_json_done}" ${RELEASE} complete
|
|
|
|
SUBJECT='Fedora '$RELEASE' compose report: '$SHORTCOMPOSE_ID' changes'
|
2017-12-04 15:44:52 -08:00
|
|
|
#for tomail in $TOMAIL ; do
|
2018-01-25 12:12:21 +01:00
|
|
|
# cat $DESTDIR/logs/*verbose $DESTDIR/logs/depcheck | \
|
2017-12-04 15:44:52 -08:00
|
|
|
# mutt -e "set from=\"$FROM\"" -e 'set envelope_from=yes' -s "$SUBJECT" $tomail
|
|
|
|
#done
|