Commit Graph

119 Commits

Author SHA1 Message Date
Adam Williamson
e82a95813c nightly scripts: Use $DESTDIR more
We set $DESTDIR in all these scripts, but then use its value
several times without just...using it. Let's use it!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-01-25 12:12:21 +01:00
Adam Williamson
93bf27c8f2 Emit 'location' key in fedmsgs, pointing to compose URL
For fedmsg_meta we have to provide a 'link'. At present we do
this with a rather large and icky pile of conditionals which
doesn't work properly. It's much simpler to figure out the URL
in the compose script and emit it in the fedmsg, then meta can
just...use it. For older fedmsgs we'll just point to the base
kojipkgs location for the link.

For synced composes, we use the synced URL on dl (this matches
what the meta stuff tried to do before). For non-synced composes
we just use the kojipkgs location.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-01-25 11:00:52 +01:00
Adam Williamson
ded7d70929 nightly: include shortname in fedmsgs, clean up version vars
The main point here is to include the compose 'shortname' in
compose fedmsgs, as #484 does for post-release compose types.
While doing this I noticed there's rather a lot of mess and
cruft in the various version-related variables here (and a few
others).

`GIT_BRANCH` variable is never used anywhere any more, so it's
removed.

`BRANCH` was never used anywhere, so it is gone. (It appears
to have been added in a25b08c7 but never used). There seemed
to be no useful distinction between `DIST` and `BRANCHED`, and
both actually seem to be used mainly to specify a release
version. The exception is that `BRANCHED` was set to
"Modular-Bikeshed" in nightly-modular.sh , but there is no
justification for this that I can find, and its only practical
use was as the value of the `branch` key in fedmsgs. So far as
I can figure out, the original purpose of the `branch` key was
for when we sent out `compose.branched.(foo)` fedmsgs - it
was e.g. 'f18' when Branched was Fedora 18. As we now don't
send out `compose.branched` messages any more, but include the
release number in the message topic for Branched messages
(e.g. `compose.27`), this purpose no longer applies. So the
key just doesn't seem to have any particular use at all any
more, and I don't think changing its value will have any real
consequences.

Thus, `BRANCH`, `BRANCHED` and `DIST` are replaced by `RELEASE`
and `RELEASE_TITLE`, which specify the release version. Their
values will be identical for numerical release versions, but
for Rawhide and Bikeshed we need to know both the lower-case
and title-case versions of the name; RELEASE_TITLE is the
title-case version.

We use `$SHORT-$RELEASE_TITLE` to replace several instances
of hardcoding in the scripts (e.g. Fedora-Modular-Bikeshed).

The `ATOMIC`, `ATOMICREPO` and `ATOMICDEST` variables that were
blindly copied from nightly.sh to nightly-modular.sh, but never
actually used in it, are also removed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-12-04 17:57:16 -08:00
Kevin Fenzi
755b505664 Merge #457 Revert "Do not block retired packages for now" 2017-11-28 18:23:18 +00:00
Mohan Boddu
b5869f393a Revert "Do not block retired packages for now"
We have a release now, so we can revert this commit and start
blocking retired packages

This reverts commit 35bc52d14b.

Signed-off-by: Mohan Boddu <mboddu@redhat.com>
2017-11-15 10:48:45 -05:00
Robert Marshall
53f131ee82 Let pungi produce comps file in rawhide composes
- change comps_file setting in the fedora.conf to use the new scm_dict
  type from the latest pungi release
- remove the make step from the nightly shell script

Signed-off-by: Robert Marshall <rmarshall@redhat.com>
2017-11-13 16:31:50 -05:00
Patrick Uiterwijk
35bc52d14b Do not block retired packages for now
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2017-11-03 19:22:15 +01:00
Till Maas
a5a1994895 Explicitly select koji profile
This is required to adjust to
https://pagure.io/releng/pull-request/7059

Signed-off-by: Till Maas <opensource@till.name>
2017-09-28 23:15:46 +02:00
Lubomír Sedlář
968f8fdf52 Wait for ostree to be signed before using it
Images should include signed bits, so we need to block the compose until
the commit is signed.

Related: https://pagure.io/pungi/issue/650
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-21 07:52:33 +02:00
Dusty Mabe
ea7b045988
nightly: prune atomic workstation
This adds pruning for atomic workstation. We don't yet rsync atomic
workstation out like we do for atomic host. I'm not sure why but didn't
want to add that as part of this commit.

Signed-off-by: Dusty Mabe <dusty@dustymabe.com>
2017-09-15 08:40:17 -04:00
Dusty Mabe
2726280e0c
nightly: use Atomic Host in variables
Signed-off-by: Dusty Mabe <dusty@dustymabe.com>
2017-09-15 08:37:56 -04:00
Lubomír Sedlář
45e48488f4 Fix broken deps e-mail header for future releases
Instead of having to add the `--treename` argument after each branching,
let the script compute it automatically.

Relates: https://pagure.io/pungi-fedora/issue/350
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-11 11:10:34 +02:00
Mohan Boddu
eec91246d4 Calling rawhide instead of branched for rawhide nightly composes
Signed-off-by: Mohan Boddu <mboddu@redhat.com>
2017-08-21 16:27:43 -04:00
Dan Horák
36994aeda7 sync s390x to fedora-secondary
Signed-off-by: Dan Horák <dan@danny.cz>
2017-08-16 10:05:36 +02:00
Lubomír Sedlář
4f15d62cdd Define start fedmsg json value
The nightly scripts try to send an announcement about compose starting.
As this happens before Pungi even started, the compose ID is not known
and can not be included in the message. The message about starting rsync
sent later will contain the ID correctly.

Without the json defined there is a traceback in the cron output, but
the errors are not logged anywhere else (but don't actually block
anything in the compose process).

Relates: https://pagure.io/pungi/issue/698
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-08 08:54:27 +02:00
Adam Miller
ae2f611637 Switch from docker to container namespace
Fedora Atomic WG has decided to move from the "docker" namespace to
the "container" namespace for the sake of making it more generic in
order to target all OCI Compliant container runtimes instead of
a specific implementation. This update changes what is necessary for
to the base image metadata as well as add a transitional second call
to handle both the docker and container namespace for block_retired.

Signed-off-by: Adam Miller <maxamillion@fedoraproject.org>
2017-04-26 17:16:16 -05:00
Bill Peck
c5bb3f0ed1 Fix bad copy/paste path.
This fixes the missing .composeinfo for secondary arches.

Signed-off-by: Bill Peck <bpeck@redhat.com>
2017-04-03 13:57:29 -04:00
Adam Miller
a5a8e39c73 Call block_retired.py with --namespace=docker
Need to have an extra call to the block_retired.py releng script in
order to also handle the docker namespace of "packages" from pkgdb

Signed-off-by: Adam Miller <maxamillion@fedoraproject.org>
2017-03-24 16:45:28 -05:00
Dan Horák
13af4ad3e6 nightly.sh: update arches for s390 koji
Signed-off-by: Dan Horák <dan@danny.cz>
2017-03-09 18:33:30 +01:00
Dan Horák
1aadaa9dca nightly.sh: update for updated srpm-excluded-arch.py
Signed-off-by: Dan Horák <dan@danny.cz>
2017-03-09 18:33:24 +01:00
Mohan Boddu
5e68322d22 Create dir if the dir not exists for syncing
Fix for one of the issues in #6676

Signed-off-by: Mohan Boddu <mboddu@redhat.com>
2017-03-04 12:44:37 -05:00
Lubomír Sedlář
b4670db118 Copy source data from compose
This will copy source packages as well when splitting the compose. All
stuff going to /pub/fedora will have sources.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-12-14 17:38:15 +01:00
Dennis Gilmore
9da45615e8 sync ppc64 and ppc64le to fedora-secondary
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-10-29 17:06:12 -05:00
Dennis Gilmore
1caf794933 add excludes for the arches not in primary koji yet
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-09-14 08:15:15 -05:00
Dennis Gilmore
9161baf281 sync the content to different locations
aarch64 and i386 to fedora-secondary
armhfp and x86_64 to alt for Cloud and Labs
everything else to /pub/fedora

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-09-13 22:09:23 -05:00
Dennis Gilmore
9782d9d36b fix up the path to srpms for finding arch exluded packages
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-09-13 21:58:46 -05:00
Lubomír Sedlář
c957262b5a If compose fails, stop running everything else
If a compose fails, we don't want to run any dependency checks nor send
any e-mails.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-08-22 11:06:18 +02:00
Lubomír Sedlář
91e76bb480 Use compose-utils to copy variants into public location
This requires compose-utils >= 0.1.7.

Currently only primary architectures are composed based on this config.
Once other arches are added, they can be filtered with `--arch` option.

To debug what will happen when the command is run, add `--dry-run`. It
will print the exact rsync commands that would be used. Appending `--
--dry-run` at the end of the command will pass the option to rsync and
give different debugging information.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-08-18 10:33:09 +02:00
Dennis Gilmore
6ef9598cd4 fix up finding the compose ID's
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-06-26 19:57:04 -05:00
Dennis Gilmore
acdcba9c91 we can not use -delete as we have to limit the depth so we do not remove files from the current composes
the rm -rf is needed to clean up the directories -delete gives us
find: cannot delete '/mnt/koji/compose/rawhide/Fedora-Rawhide-20160512.n.0/compose': Directory not empty
find: cannot delete '/mnt/koji/compose/rawhide/Fedora-Rawhide-20160512.n.0/logs': Directory not empty

Revert "Use -delete instead of -exec rm it's quicker because it doesn't fork for every file"

This reverts commit 1b80cae931.

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-05-27 14:20:40 -05:00
Peter Robinson
1b80cae931 Use -delete instead of -exec rm it's quicker because it doesn't fork for every file
Signed-off-by: Peter Robinson <pbrobinson@fedoraproject.org>
2016-05-20 11:42:17 +01:00
Peter Robinson
b3a56aee17 Drop MOCK variable
No longer used now atomic is composed in pungi

Signed-off-by: Peter Robinson <pbrobinson@fedoraproject.org>
2016-05-18 15:18:59 +01:00
Dennis Gilmore
5f9f66a2ca go about deleting the files differently
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-05-16 20:35:20 -05:00
Dennis Gilmore
b81d96f5d4 clean up rawhide compose older than 2 weeks old
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-05-16 19:46:41 -05:00
Dennis Gilmore
1edd5f8bbe prune the aomic repo before syncing
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-05-16 19:37:41 -05:00
Dennis Gilmore
91aca2552c fix up announcing compose start
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-05-16 16:00:44 -05:00
Dennis Gilmore
7cdfd1d583 change ATOMICDEST to non mirrored location
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-05-16 14:39:41 -05:00
Dennis Gilmore
10f81a2317 fix up comps generation
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-05-14 08:37:21 -05:00
Kevin Fenzi
e249ad33f1 Handle move of comps and spin-kickstarts to pagure
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2016-05-13 16:32:43 -06:00
Kevin Fenzi
1a2692bdaf Set DESTDIR as it's used to check deps and some other items.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2016-04-27 09:53:24 -06:00
Ralph Bean
fa6f62ac25 Send compose_id along with the rsync fedmsg.
Signed-off-by: Ralph Bean <rbean@redhat.com>
2016-04-25 12:24:57 -04:00
Dennis Gilmore
57fef86770 set the name in .composeinfo to be the compose id
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-04-08 08:34:04 -05:00
Dennis Gilmore
764ba668a9 remove the bolt on atomic hack
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-31 17:59:09 -05:00
Dennis Gilmore
5abfff571b we need to remove the old composeinfo files in order to make a new one
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-18 15:00:37 -05:00
Dennis Gilmore
c688bf345e make old style .composeinfo files on the master mirror for beaker import
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-18 14:52:10 -05:00
Dennis Gilmore
211be10ea4 change the url for the fedora-atomic.git to pagure
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-18 14:34:57 -05:00
Dennis Gilmore
14e980489f checkout spin-kickstarts
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-14 17:55:57 -05:00
Peter Robinson
84645b1bd4 fix minor typo in comment
Signed-off-by: Peter Robinson <pbrobinson@fedoraproject.org>
2016-03-14 12:35:14 +00:00
Dennis Gilmore
95b4299b66 sync teh atomic repo
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-11 18:49:25 -06:00
Ralph Bean
d136a4cc87 Just re-use the fedmsg definition from the releng repo.
Signed-off-by: Ralph Bean <rbean@redhat.com>
2016-03-03 14:05:48 -05:00
Ralph Bean
e23a5f736d Some start/stop fedmsg messages wrapped around nightly.sh.
As well as fedmsg notify on rsync start/stop.

Signed-off-by: Ralph Bean <rbean@redhat.com>
2016-03-03 13:52:29 -05:00
Dennis Gilmore
b519942d4c attempt to send each email inidividulaly
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-03 09:18:36 -06:00
Dennis Gilmore
91b28be08f use BRANCHED not VERSION in atomic installer
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-03-02 17:46:26 -06:00
Dennis Gilmore
9dc2ae89a9 tie in missing functionality
tie in some scripts for the releng repo and mash to get thinsg more complete

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-29 16:22:42 -06:00
Dennis Gilmore
4acb824671 remove leading + from SUBJECT
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-28 22:29:17 -06:00
Dennis Gilmore
2db6ecc7f5 change the email subject
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-28 22:26:45 -06:00
Dennis Gilmore
6a498f572c make sure we send the email to the lists
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-28 22:20:37 -06:00
Dennis Gilmore
9e080cdb92 use TARGET_DIR variables and not hardcoded values
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-27 08:49:01 -06:00
Dennis Gilmore
3e2651f371 set RSYNCPREFIX so we sudo when needed
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-26 21:22:39 -06:00
Dennis Gilmore
1d86df06a0 sync the compose bits to the master mirror
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-26 21:20:38 -06:00
Dennis Gilmore
cbaae50667 cat not echo the COMPOSE_ID files
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-23 21:51:33 -06:00
Dennis Gilmore
a25b08c727 attempt to make a atomic repo and installer
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2016-02-23 16:34:37 -06:00
Dennis Gilmore
7d1294ffd7 fixup a few wrong settings in the configs
add i386 images

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-12-10 10:14:08 -06:00
Dennis Gilmore
49c78ea88d add missing f in the comps file
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-12-08 09:31:42 -06:00
Dennis Gilmore
19e8689534 grab and make the latest comps file
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-12-08 09:15:29 -06:00
Dennis Gilmore
14fba4cef7 set teh script to use for fedmsg notifications
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-12-04 13:58:19 -06:00
root
41d8d5262e update the shell script based on suggestions from dmach
Signed-off-by: root <root@compose-x86-01.phx2.fedoraproject.org>
2015-07-24 13:13:51 -05:00
Dennis Gilmore
778eabff0e updates that have been done for testing
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-07-07 10:16:29 -05:00
Dennis Gilmore
ced9341be4 setup the initial fedora repo
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-05-29 15:04:00 -05:00