forked from fedora-copr/copr
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathcopr-backend.spec
1708 lines (1483 loc) · 67 KB
/
copr-backend.spec
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
%if 0%{?rhel} <= 7 && 0%{?rhel} > 0
%global _pkgdocdir %{_docdir}/%{name}-%{version}
%endif
%global prunerepo_version 1.20
%global tests_version 5
%global tests_tar test-data-copr-backend
%global copr_common_version 0.25.1~~dev0
Name: copr-backend
Version: 2.0
Release: 1%{?dist}
Summary: Backend for Copr
License: GPL-2.0-or-later
URL: https://github.com/fedora-copr/copr
# Source is created by:
# git clone %%url && cd copr
# tito build --tgz --tag %%name-%%version-%%release
Source0: %{name}-%{version}.tar.gz
Source1: https://github.com/fedora-copr/%{tests_tar}/archive/v%{tests_version}/%{tests_tar}-%{tests_version}.tar.gz
BuildArch: noarch
BuildRequires: asciidoc
BuildRequires: createrepo_c >= 0.16.1
BuildRequires: libappstream-glib-builder
BuildRequires: libxslt
BuildRequires: make
BuildRequires: redis
BuildRequires: rsync
BuildRequires: systemd
BuildRequires: util-linux
BuildRequires: python3-devel
BuildRequires: python3-setuptools
BuildRequires: python3-copr
BuildRequires: python3-copr-common >= %copr_common_version
BuildRequires: python3-daemon
BuildRequires: python3-dateutil
BuildRequires: python3-distro
BuildRequires: python3-gobject
BuildRequires: python3-httpretty
BuildRequires: python3-humanize
BuildRequires: python3-munch
BuildRequires: python3-netaddr
BuildRequires: python3-packaging
BuildRequires: python3-pytest
BuildRequires: python3-pytz
BuildRequires: python3-requests
BuildRequires: python3-resalloc
BuildRequires: python3-retask
BuildRequires: python3-setproctitle
BuildRequires: python3-sphinx
BuildRequires: python3-tabulate
BuildRequires: python3-zstandard
BuildRequires: python3-cachetools
BuildRequires: modulemd-tools >= 0.6
BuildRequires: prunerepo >= %prunerepo_version
BuildRequires: dnf
Requires: (copr-selinux if selinux-policy-targeted)
Requires: ansible
Suggests: awscli
Requires: createrepo_c >= 0.16.1
Requires: crontabs
Requires: gawk
Requires: libappstream-glib-builder
Requires: lighttpd
Recommends: logrotate
Requires: mock
Requires: obs-signd
Requires: openssh-clients
Requires: prunerepo >= %prunerepo_version
Requires: python3-copr
Requires: python3-copr-common >= %copr_common_version
Recommends: python3-copr-messaging
Requires: python3-daemon
Requires: python3-dateutil
Recommends: python3-fedmsg
Requires: python3-gobject
Requires: python3-humanize
Requires: python3-jinja2
Requires: python3-munch
Requires: python3-netaddr
Requires: python3-novaclient
Requires: python3-packaging
Requires: python3-pytz
Requires: python3-requests
Requires: python3-resalloc >= 3.0
Requires: python3-retask
Requires: python3-setproctitle
Requires: python3-tabulate
Requires: python3-boto3
Requires: python3-cachetools
Requires: redis
Requires: rpm-sign
Requires: rsync
Requires: modulemd-tools >= 0.6
Recommends: util-linux-core
Requires: zstd
Requires(post): systemd
Requires(preun): systemd
Requires(postun): systemd
%description
COPR is lightweight build system. It allows you to create new project in WebUI,
and submit new builds and COPR will create yum repository from latest builds.
This package contains backend.
%package doc
Summary: Code documentation for COPR backend
%description doc
COPR is lightweight build system. It allows you to create new project in WebUI,
and submit new builds and COPR will create yum repository from latests builds.
This package include documentation for COPR code. Mostly useful for developers
only.
%prep
%setup -q -a 1
%build
make -C docs %{?_smp_mflags} html
%py3_build
%install
%py3_install
install -d %{buildroot}%{_sharedstatedir}/copr/public_html/results
install -d %{buildroot}%{_pkgdocdir}/lighttpd/
install -d %{buildroot}%{_sysconfdir}/copr
install -d %{buildroot}%{_sysconfdir}/logrotate.d/
install -d %{buildroot}%{_unitdir}
install -d %{buildroot}/%{_var}/log/copr-backend
install -d %{buildroot}/%{_var}/run/copr-backend/
install -d %{buildroot}/%{_tmpfilesdir}
install -d %{buildroot}/%{_sbindir}
install -d %{buildroot}%{_sysconfdir}/cron.daily
install -d %{buildroot}%{_sysconfdir}/cron.weekly
install -d %{buildroot}%{_sysconfdir}/sudoers.d
install -d %{buildroot}%{_bindir}/
cp -a copr-backend-service %{buildroot}/%{_sbindir}/
cp -a run/* %{buildroot}%{_bindir}/
cp -a conf/copr-be.conf.example %{buildroot}%{_sysconfdir}/copr/copr-be.conf
install -p -m 755 conf/crontab/daily %{buildroot}%{_sysconfdir}/cron.daily/copr-backend
install -p -m 755 conf/crontab/weekly %{buildroot}%{_sysconfdir}/cron.weekly/copr-backend
cp -a conf/lighttpd/* %{buildroot}%{_pkgdocdir}/lighttpd/
cp -a conf/logrotate/* %{buildroot}%{_sysconfdir}/logrotate.d/
cp -a conf/tmpfiles.d/* %{buildroot}/%{_tmpfilesdir}
# for ghost files
touch %{buildroot}%{_var}/log/copr-backend/copr.log
touch %{buildroot}%{_var}/log/copr-backend/prune_old.log
cp -a units/*.{target,service} %{buildroot}/%{_unitdir}/
install -m 0644 conf/copr.sudoers.d %{buildroot}%{_sysconfdir}/sudoers.d/copr
install -d %{buildroot}%{_sysconfdir}/logstash.d
install -d %{buildroot}%{_datadir}/logstash/patterns/
cp -a conf/logstash/lighttpd.pattern %{buildroot}%{_datadir}/logstash/patterns/lighttpd.pattern
install -d %{buildroot}%{_pkgdocdir}/examples/%{_sysconfdir}/logstash.d
cp -a conf/logstash/copr_backend.conf %{buildroot}%{_pkgdocdir}/examples/%{_sysconfdir}/logstash.d/copr_backend.conf
cp -a docs/build/html %{buildroot}%{_pkgdocdir}/
%check
./run_tests.sh -vv --no-cov
%pre
getent group copr >/dev/null || groupadd -r copr
getent passwd copr >/dev/null || \
useradd -r -g copr -G lighttpd -s /bin/bash -c "COPR user" copr
/usr/bin/passwd -l copr >/dev/null
%post
%systemd_post copr-backend.target
%preun
%systemd_preun copr-backend.target
%postun
%systemd_postun_with_restart copr-backend-log.service
%systemd_postun_with_restart copr-backend-build.service
%systemd_postun_with_restart copr-backend-action.service
%files
%license LICENSE
%python3_sitelib/copr_backend
%python3_sitelib/copr_backend*egg-info
%dir %{_sharedstatedir}/copr
%dir %attr(0755, copr, copr) %{_sharedstatedir}/copr/public_html/
%dir %attr(0755, copr, copr) %{_sharedstatedir}/copr/public_html/results
%dir %attr(0755, copr, copr) %{_var}/run/copr-backend
%dir %attr(0755, copr, copr) %{_var}/log/copr-backend
%ghost %{_var}/log/copr-backend/*.log
%config(noreplace) %{_sysconfdir}/logrotate.d/copr-backend
%dir %{_pkgdocdir}
%doc %{_pkgdocdir}/lighttpd
%dir %{_sysconfdir}/copr
%config(noreplace) %attr(0640, root, copr) %{_sysconfdir}/copr/copr-be.conf
%{_unitdir}/*.service
%{_unitdir}/*.target
%{_tmpfilesdir}/copr-backend.conf
%{_bindir}/*
%{_sbindir}/*
%config(noreplace) %{_sysconfdir}/cron.daily/copr-backend
%config(noreplace) %{_sysconfdir}/cron.weekly/copr-backend
%{_datadir}/logstash/patterns/lighttpd.pattern
%config(noreplace) %attr(0600, root, root) %{_sysconfdir}/sudoers.d/copr
%files doc
%license LICENSE
%doc
%{_pkgdocdir}/
%exclude %{_pkgdocdir}/lighttpd
%changelog
* Wed Oct 02 2024 Jiri Kyjovsky <[email protected]> 2.0-1
- Implement project, build, chroot deletion and creation in Pulp
- Fix timeout fail-safe in case copr-rpmbuild hangs up
- Remove fake jobs and pass specific arguments to storage calls
- Add support for Pulp
- Improve logging for expiring user SSH builders
* Tue May 21 2024 Jakub Kadlcik <[email protected]> 1.177-1
- Multiple attempts to create repository before giving up
- Run rawhide-to-release for all CoprDirs
- Remove static methods from tests
* Sat Mar 16 2024 Pavel Raiskup <[email protected]> 1.176-1
- print user SSH instructions before the build starts
* Mon Mar 04 2024 Pavel Raiskup <[email protected]> 1.175-1
- call correct (renamed) _discard_running_worker() internal method
* Fri Mar 01 2024 Pavel Raiskup <[email protected]> 1.174-1
- allow user SSH to builders
- drop ActionResult and use BackendResultEnum from copr-common
- replace backend-specific ActionType with ActionTypeEnum from copr-common
- limit the stdout/stderr of ssh.run_expensive() commands
- backend now periodically checks if the resalloc ticket isn't failed
- keep logs 6 weeks instead of 13
- rename dispatcher scripts
* Thu Nov 23 2023 Pavel Raiskup <[email protected]> 1.173-1
- enforce createrepo_c gzip compression (f39+ switched to zstd)
- self-identify the resalloc resource in logs
- dropping the documentary playbooks from copr-backend payload
- nicer unknown-resalloc-tickets output
- worker to not call keygen for source builds at all
- don't sign products of srpm-build
- longer timeout for fallback generating GPG keys after build
- recreate missing repodata so that prunerepo doesn't traceback
- use the rename HashWorkerLimit instead of GroupWorkerLimit
- provide per-arch & per-owner worker limit implemented
- collect and compress fedora-review logs after run
- react on staled SSH connections in some cases
* Tue Aug 15 2023 Pavel Raiskup <[email protected]> 1.172-1
- dump the /update/ payload to worker.log
- don't run external command(s) to collect built packages
- don't eat the "build detail collecting" traceback
- fixes in the unknown-resalloc-tickets.py helper
- more careful format_evr() method
- fix tests for zst compression on F39+
- log task dict in case of error returned from redis
- skip builds for ExcludeArch and "not" ExclusiveArch
- offload NEVRA (s)rpm parsing to copr-rpmbuild
- redis authentication support added
* Tue Jun 06 2023 Pavel Raiskup <[email protected]> 1.171-1
- copr_prune_results.py: work-around the arg_max problem
* Mon Jun 05 2023 Pavel Raiskup <[email protected]> 1.170-1
- copr_prune_results.py: don't enforce appstream-builder, ask FE
- copr_prune_results.py: just one API call to FE per one repo
* Tue May 23 2023 Jakub Kadlcik <[email protected]> 1.169-1
- Forking: better handle FileExistsError
- Run the copr-rpmbuild command with task URL, not build ID
* Wed Apr 05 2023 Jiri Kyjovsky <[email protected]> 1.168-1
- Bump version for release mess
* Tue Apr 04 2023 Jiri Kyjovsky <[email protected]> 1.167-1
- Run createrepo without --database
- Make copr_messaging optional
* Wed Mar 22 2023 Jiri Kyjovsky <[email protected]> 1.166-1
- Don't include package name into srpm result dir name
- Remove libmodulemd1 dependency
- Hardlink RPMs while doing rawhide_to_release
- Make sign key domain name configurable
* Wed Jan 25 2023 Jakub Kadlcik <[email protected]> 1.165-1
- Skip the test_run_prunerepo test because of Koji
* Tue Jan 24 2023 Jakub Kadlcik <[email protected]> 1.164-1
- Fix chroot version parsing with new python-packaging
- Fix traceback for non-existing tasks
- Python: drop the unneeded marshmallow dep
- Log general exceptions to find issues more easily
- Proper log argument formatting instead of .format
- Use spdx license
* Wed Nov 30 2022 Pavel Raiskup <[email protected]> 1.163-1
- start systemd services after the redis.service
- build worker - list the built RPMs with rpm --nosignature
* Sat Nov 26 2022 Jakub Kadlcik <[email protected]> 1.162-1
- use OpenPGP v4 signatures
- migrate from pipes to shlex
- require redis.service to be started
- move to GitHub home page
- add resultdir cleaner
- move dispatcher and background workers to copr-common
- de-prio IO for the analyze-results script
- don't traceback when there are no files in the S3 storage
- allow devel instance to remove access files
- send non-CDN hitcounter hits in chunks
- copr-backend-unknown-resalloc-tickets script
- work with multiple CDN hostnames per instance
- move setup_script_logger to copr-common
* Mon Sep 26 2022 Pavel Raiskup <[email protected]> 1.161-1
- sign everything EPEL-5+ with sha256 hashalgo
* Tue Sep 20 2022 Jakub Kadlcik <[email protected]> 1.160-1
- aws-hitcounter: remove temporary files as soon as possible
- aws-hitcounter: ignore downloaded SRPM files
- aws-hitcounter: decode special characters from URLs
* Tue Aug 16 2022 Pavel Raiskup <[email protected]> 1.159-1
- count only hits from an appropriate CDN hostname
- add option for infinite number of attempts to the hitcounter script
- print more reasonable output from AWS hitcounter script
* Tue Aug 16 2022 Jiri Kyjovsky <[email protected]> 1.158-1
- log every request that is sent to frontend
* Tue Jul 26 2022 Jakub Kadlcik <[email protected]> 1.157-1
- Don't use --keep-all-metadata
- Search for comps.xml in chroot dir
* Tue Jun 21 2022 Jakub Kadlcik <[email protected]> 1.156-1
- Consolidate the two hitcounter scripts
- Dump Resalloc ticket ID and hostname to backend.log
- Automatically restart services in %%post
- Don't count RPMs downloaded from Mock
- Attempt to sign multiple times
- Try multiple attempts of creating GPG keys
* Mon Apr 04 2022 Pavel Raiskup <[email protected]> 1.155-1
- fix slow priority queue filling
- speedup the background-process spawner
* Fri Mar 18 2022 Pavel Raiskup <[email protected]> 1.154-1
- copr_fix_gpg: automatically refresh CDN cache
- copr_fix_gpg: don't use --skip-stat for copr-repo when RPMs are re-signed
- action processor: properly return failures of the fork action
- sign EL8+ with sha256 hash algorithm
- copr_fix_gpg: add a new --chroot option
- copr_fix_gpg: skip non-chroot directories
- add hitcounter script for AWS CDN
- backend: tasks in concurrent sandboxes reprioritized to be more fair
* Wed Feb 02 2022 Silvie Chlupova <[email protected]> 1.153-1
- less aggressive final_prunerepo setting
- analyze-results: dump data for projects' chroots
- basic build tagging
- better "regenerate repo" instructions
- limit RubyGems and PyPI package names length
- Disable coverage analysis during RPM build
- python code for removing unused tarballs on dist-git server
* Wed Nov 10 2021 Silvie Chlupova <[email protected]> 1.152-1
- Fixup ACR handling
- Drop the unused pid file from the specfile
* Thu Sep 30 2021 Silvie Chlupova 1.151-1
- backend: use lock(timeout=5) to work-around fair-locks
* Wed Aug 25 2021 Pavel Raiskup <[email protected]> 1.150-1
- request arch_noarch resalloc tag for source RPM builds
- re-try /bin/sign call upon connection timeout
- drop 'check_consecutive_build_fails' script
* Tue Jun 15 2021 Pavel Raiskup <[email protected]> 1.149-1
- new weekly cron-job for analyzing storage use (graphs, statistics)
- added some convenience globals into copr_backend.setup module
- keep the max batch size really on the MAX_IN_BATCH limit
- require up2date rpmbuild version
- new action for removing CoprDirs (triggered by cron on frontend)
- fix the CompsUpdate action
- users now can disable appstream metadata generation themselves (without admins asistance)
- handle results.json given by builder, and provide it to frontend (served as APIv3 later)
- pruner: allow pruning also the finalized chroots on demand
- invent FE-BE API version, so backend politely waits for an updated copr-frontend version
* Thu May 13 2021 Pavel Raiskup <[email protected]> 1.148-1
- work with builders also over ipv6
* Sun May 02 2021 Pavel Raiskup <[email protected]> 1.147-1
- fix logging traceback for module builds
- call creatrepo_c with --update if possible
- don't do full createrpeo with --rpms-to-remove
* Fri Apr 30 2021 Pavel Raiskup <[email protected]> 1.146-1
- better fixes for the appstream-builder generated files
- new helper script named /bin/copr-assure-permissions
- do not rsync-copy permissions from the builder
- log the partial "prunerepo" effects into resultdir
- better, more verbose call_copr_repo logging
* Tue Apr 27 2021 Jakub Kadlcik <[email protected]> 1.145-1
- backend: make the walk_limited test not dependend on its output order, pt2
* Tue Apr 27 2021 Jakub Kadlcik <[email protected]> 1.144-1
- backend: make the walk_limited test not dependend on its output order
* Tue Apr 27 2021 Jakub Kadlcik <[email protected]> 1.143-1
- backend: fix copr_prune_results logging once more
- backend: better logging in prunerepo
- backend: prunerepo: don't re-createrepo when no rpm is removed
- backend: catch correct client exceptions in copr_prune_results
- test: backend: change prunerepo logic, use get_rpms_to_remove from prunerepo
- backend: new fixture for testing prunerepo
- backend: use safe defaults if APIv3 result doesn't contain what it should
- backend: migrate to APIv3
- backend: better how-to-redirect logs comment
- backend, frontend, keygen, distgit: keep cca 3 months of logs
- backend: don't createrepo in srpm-builds on delete
- backend: test walk_limited function from helpers
* Tue Mar 16 2021 Pavel Raiskup <[email protected]> 1.142-1
- prepare for the centos-stream-8 rename
- add script to prune srpm-build directories
- pruner: correctly deliver the final prunerepo stamp to frontend
- pruner: logging through RedisLogHandler
- pruner: better parallelization
- pruner: re-try be-fe communication upon failures
- require up2date modulemd-tools
* Wed Jan 20 2021 Pavel Raiskup <[email protected]> 1.141-1
- run prunerepo in parallel
- add one-shot copr_find_wrong_chroot_artifacts.py script
- support modulemd v2
* Tue Dec 01 2020 Pavel Raiskup <[email protected]> 1.140-1
- fix frontend-client post arguments
* Mon Nov 30 2020 Pavel Raiskup <[email protected]> 1.139-1
- require appropriate common version
- use common for repeatedly sends requests to frontend
* Mon Nov 30 2020 Pavel Raiskup <[email protected]> 1.138-1
- get back to using standard createrepo_c command from createrepo_mod
- drop call to stomp's conn.start() (it was dropped)
* Mon Nov 09 2020 Jakub Kadlcik <[email protected]> 1.137-1
- backend: test action for deleting chroot
- backend: fix testsuite for the new createrepo_c
- frontend: don't set ended_on for canceled builds
- all: run pytest with -vv in package build
- common, cli, python, rpmbuild, frontend, backend: DistGit source method
- backend: use createrepo_mod tool for generating module repodata
* Wed Aug 12 2020 Pavel Raiskup <[email protected]> 1.136-1
- testsuite: give more time to the slow Koji builders
* Mon Aug 10 2020 Pavel Raiskup <[email protected]> 1.135-1
- prioritize all non-background jobs
- fix up libmodulemd dependency
* Fri Jun 19 2020 Pavel Raiskup <[email protected]> 1.134-1
- fix copr-repo to work with absolute paths
- automatically batch the createrepo requests
- scheduler is now fair, and ordered
- indefinitely retry workers' talk to frontend
- allow canceling also "starting" builds
- more verbose delete action in logs
- cleanup the example configuration
- use FileHandler for backend.log, fixes traceback
* Tue Jun 09 2020 Pavel Raiskup <[email protected]> 1.133-1
- better build task priority processing
- dump attempt to send message to backend.log
- drop the VMM concept, replaced with resalloc
- delegate more work to the builder code
- external blob tarball for unittests
- buggy error handler in pkg_name_evr()
- basic build task priority
- the reschedule-all builds idiom removed
- fix the build cancelation
- drop duplicate BuildRequire on python-requests
- require the newest version of copr-common
- minimalize the transfered amount of information about actions from FE
- process actions in regard to their priority
- move backend's code to standard PYTHONPATH
- move ActionResult to copr_common.enums
- actions/builds use the same WorkerManager logic
- more verbose rawhide to release action processing
* Wed Feb 05 2020 Pavel Raiskup <[email protected]> 1.132-1
- better handle invalid options in copr-repo --add/--delete
- copr-repo: optimize-out useless createrepo_c runs
- move initial createrepo check from dispatcher to worker
- don't send messages on bus N-times
- /bin/copr-repo now respects .disable-appstream files
- drop unused build_deleting_without_createrepo option
* Wed Jan 15 2020 Tomas Hrnciar <[email protected]> 1.131-1
- put build-ID.log file to resultdir
- call call_copr_repo if initial createrepo failed
- Build Dispatcher does not wait forever till repo is created,
it creates it manually
- properly delete logs for old builds
- delete build-ID.log files again
- edit repositories only by new 'copr-repo' tool
- fix multi-build delete
- fix for not saving end time of actions
- lower traffic in build_dispatcher log
- more resilient redis logging
- attempt to publish on msgbus N-times
- log service: move RequiredBy to [Install]
- keep worker ID in proc title
* Fri Dec 06 2019 Pavel Raiskup <[email protected]> 1.130-1
- backend: execute actions with sane umask=0022
* Wed Dec 04 2019 Pavel Raiskup <[email protected]> 1.129-1
- do not start a build if copr_base is not available yet
- systemd services' restart re-ordering
- de-duplicate frontend_.update() call when reattaching to existing build
- allow specifying timeout for spawn/terminate playbooks
- removing dependecy on euca2ools in spec
- send `uses_devel_repo' as a part of task info
- correctly configure logrotate
- get_redis_logger: skip log entries bellow log_level
- delete leftover action workers from redis
* Fri Oct 11 2019 Pavel Raiskup <[email protected]> 1.128-1
- restart copr-backend sub-services on failure
- don't kill action processors by 'systemctl restart'
- lower the log traffic in build_dispatcher.log
* Thu Oct 03 2019 Pavel Raiskup <[email protected]> 1.127-1
- fix testsuite for slow Koji builders
* Thu Oct 03 2019 Pavel Raiskup <[email protected]> 1.126-1
- more reliable communication with frontend (#1021)
- only ask for auto_createrepo once per project
- parallel handling of actions (#1007)
- don't provide builder-live.log once the build ended, and
add 'copr-compress-live-logs' helper (#985)
- less exceptions in logs
- project forking fixes
- depend on copr-messaging, not fedora-messaging
- fixes for copr_print_results_to_delete.py script
* Wed Aug 28 2019 Dominik Turecek <[email protected]> 1.125-1
- minimize redis traffic for looping over pending-jobs (issue#902)
- batch delete builds into a single action (issue#688)
- admin opt-out createrepo after build-deleting
- fix wrong message validation class
- refine cleanup_vm_nova.py
- depend on copr-messaging
* Mon Jul 29 2019 Pavel Raiskup <[email protected]> 1.124-1
- run createrepo immediately, don't wait for build (issue#833)
- compress backend-live.log by calling gzip (issue#86)
- use copr-messaging module for validating outgoing messages
- don't run appstream-builder for PR dirs
- don't run createrepo for srpm directories
- skip VMs with failing live-check in scheduler
- sandbox builds per user/submitter/project
- drop unused compat code for droped /bin/copr-builder
- do not call appstream builder with --max-threads (issue#717)
- added copr_print_results_to_delete.py script, should help
us with removal of orphaned resources on backend storage (issue#712)
- allow disabling appstream builder per project (issue#738)
- tabular output from copr_get_vm_info.py
* Wed Apr 24 2019 Jakub Kadlčík <[email protected]> 1.123-1
- clean data for failed builds; fix #619
- replace runnecessary regex with str.endswith
- move clean_copr from prunerepo to our codebase
- cleanup_vm_nova.py: use yaml.safe_load
- don't rely on createrepo from prunerepo
- simplify logging through redis
- run sign command without sudo to fix #636
- encode 'msg' in LogRecord sooner
- fix charset warnings on redis-py v3
- fix default arguments in redis scripts
- don't prunerepo too old directories
- LogHandler: don't drop exc_info from LogRecord
- use the correct data for rawhide_to_release createrepo
- make copr_prune_results skip already pruned outdated chroots
- require libmodulemd in at least 1.7.0
- remove dependency on python3-configparser
* Mon Feb 11 2019 Jakub Kadlčík <[email protected]> 1.122-1
- Add requires python3-novaclient
- Set the architecture for which the module has been built
- Generate module artifacts in the correct format
- Compress the modules.yaml file
* Fri Jan 11 2019 Miroslav Suchý <[email protected]> 1.121-1
- remove data from outdated chroots
* Thu Jan 10 2019 Miroslav Suchý <[email protected]> 1.120-1
- update list of copr services
- Use oslo_concurrency for craeterepo locking
- use run_cmd() in pkg_name_evr()
- drop "downloading" state
- allow blacklisting packages from chroots
* Fri Oct 19 2018 Miroslav Suchý <[email protected]> 1.119-1
- optimize copr_log_hitcounter.py for speed a bit
- move selinux rules to copr-selinux
- fix traceback for non-serializable log message
- fix tracebacks in copr-backend-log
- more robust run_tests.sh
- remove unused imports
- py3 compat for msgbus support
- use git_dir_archive instead of git_dir_pack
- migrate from deprecated python3-modulemd to libmodulemd
- doc: remove warning that _static directory does not exists
- doc: the undeline need to be at least as long as the title
* Thu Aug 23 2018 clime <[email protected]> 1.118-1
- fix logging exception
- send proper arguments for rawhide_to_release
- packaging: Python 2/3, RHEL/Fedora fixes
* Mon Aug 06 2018 clime <[email protected]> 1.117-1
- None task protection
- pagure integration
- use manual .pyc file generation
- remove unused imports, ad. pr#327
- resolving pylint warnings
- for py3 use unittest.mock
- fix msgbus ContentType to application/json
* Fri May 18 2018 clime <[email protected]> 1.116-1
- fix #291 forks are incomplete
- log more information about incoming actions
- preparation for opensuse-leap-15.0-x86_64
* Thu Apr 26 2018 Dominik Turecek <[email protected]> 1.115-1
- rpkg deployment into COPR - containers + releng continuation
- fix pagure bugs #269, #273, #221 and #268
- cleanup in test_helpers, one test added
- change order of args in StrictRedis call
- add comment about expected usage of the copr_log_hitcounter script
- try to send hit data to frontend several times from
copr_log_hitcounter
* Mon Feb 26 2018 clime <[email protected]> 1.114-1
- add possibility for copr_log_hitcounter to ignore multiple subnets
* Fri Feb 23 2018 clime <[email protected]> 1.113-1
- original builder deprecation
- remove Group tag
* Mon Feb 19 2018 clime <[email protected]> 1.112-1
- Shebangs cleanup
- escapes in changelogs
* Sun Feb 18 2018 clime <[email protected]> 1.111-1
- use netaddr instead of IPy module
- sleep after each load_jobs iteration
- python3 conversion
- UMB: adding content type
- add source_status field for Builds
- generate module artifacts rpms
- the rsync log is actually renderred directly into result dir now
- mockchain.log renamed to backend.log
- pg#192 missing records in mockchain.log
- enable running tests in spec file
- enable and update vmmamanger tests, fix three minor bugs in the
manager
- frontend now presents the whole job queue state to
backend
- copy only module builds into the repo directory
* Wed Dec 20 2017 clime <[email protected]> 1.110-1
- exception handling for hit counting when IP address cannot be parsed
* Mon Dec 18 2017 Dominik Turecek <[email protected]> 1.109-1
- terminate also 'in_use' builders if health checks have failed
- make --detached the last arg for copr-rpmbuild
- update copr_log_hitcounter to check ip against ignored pattern
- new msg bus options
- disable DNF makecache timer/service
- fix message duplication for multi-bus scenario
* Thu Nov 16 2017 Miroslav Suchý <[email protected]> 1.108-1
- optimize createrepo_c
- Revert "[backend] remove --ignore-lock from createrepo_c"
* Thu Nov 09 2017 clime <[email protected]> 1.107-1
- kill all processes in copr-rpmbuild's process group
- add --drop-resultdir switch to copr-rpmbuild call
- release_vm immediately after VM is no longer needed
- remove --ignore-lock from createrepo_c
* Wed Oct 18 2017 clime <[email protected]> 1.106-1
- run copr-rpmbuild with --verbose option
* Wed Sep 27 2017 clime <[email protected]> 1.105-1
- remove uneeded yum dep
* Tue Sep 26 2017 clime <[email protected]> 1.104-1
- update copr-rpmbuild command for the new options
- change arguments to build_id and chroot
- #128 AppStream data collection vetoes addons
- fix rpm download stats collection
- module-stuff update
* Fri Sep 15 2017 clime <[email protected]> 1.103-1
- update fedora image version to 26
- fixes for recent code
* Thu Sep 07 2017 clime <[email protected]> 1.102-1
- srpms are now being built from upstream on builders
* Wed Jun 14 2017 clime <[email protected]> 1.101-1
- remove unused helpers.run_ssh + function spacing fixup
- cancel-build action fix
* Fri Jun 09 2017 clime <[email protected]> 1.100-1
- extend check for a builder package present on a builder machine
- arbitrary dist-git branching
- remove --add-cache-id from appstream-builder call, see Bug 1426166
- change to using a standalone builder package
* Wed May 03 2017 clime <[email protected]> 1.99-1
- missing on_success_build call added back to sign packages and recreate repo after each build
* Mon Apr 24 2017 clime <[email protected]> 1.98-1
- Bug 1444804 - Logs are not present for failed builds
* Wed Apr 19 2017 clime <[email protected]> 1.97-1
- do not condrestart optional logstash service
- standalone builder option
- build reattaching after copr-backend(-build) service restart
- live mockchain log
- use openssh instead of paramiko
- update cleanup_vm_nova script
- remove buggy logging
- removed Sphinx as a dependency...
- verbose log everything we have about failed playbook
- replace fedorahosted links
- make systemd services out of ActionDispatcher and BuildDispatcher
* Thu Jan 26 2017 clime <[email protected]> 1.96-1
- Fixes for building COPR Backend and Dist-git on EL7
- simplified/improved logging of exceptions mainly
- don't use sha256 checksum for rhel-5* repos, too
- drop mentions of the max_builds_per_vm optoin
- switched usage of deprecated ansible Runner for python-paramiko module
- os_nova filter plugin fixed for python-novaclient 3
- support for STOMP msg buses
- fix Bug 1402689 regarding job cancellation
- jobgrab service is no more
- respect 'do_sign' option when forking
- fix buildroot_cmd for rhel mock profiles
* Thu Dec 01 2016 clime <[email protected]> 1.95-1
- use buildroot_pkgs substitution type according to job.chroot
- use timeout command to respect timeout param coming from frontend
- don't ship unitfiles in %%bindir
- move createrepo to the end of the rawhide_to_release handler
- modulemd 1.0.2 compatibility
- Bug 1397119 - Error reading SSH protocol banner
- added auto-prune project's option
- Bug 1086139 - [RFE] provide UI to cancel a build
- Fix misleading debug statement
- fix exception logging in ensure_dir_exists helper
- Fix chroot_setup_cmd regex for custom chroot
* Mon Sep 19 2016 clime <[email protected]> 1.94-1
- also provide default version and release for generated modules.json
* Mon Sep 19 2016 clime <[email protected]> 1.93-1
- fix NameError: global name 'result' is not defined
- fix exception logging
- Modularity support
- Bug 1357564 - RFE: allow downloading of mock profiles (reproducible builds)
- "safer" exception handling for actions
* Mon Aug 15 2016 clime <[email protected]> 1.92-1
- wrap feedback about actions to frontend into try-except
- log even the traceback from forking
- use makedirs instead of mkpath in fork action
- if anything bad happens, log exception in generate_gpg_key action
- also restart copr-backend-vmm and copr-backend-log when (re)installing
- Bug 1361344 - RFE: Allow denial of build deletion and resubmitting at project or group level
- catch errors in fork action
- set action result for comps.xml and module_md.yaml file deletion
- backend fork action now takes care of new gpg-key generation instead of frontend
- removed no longer supported --api-version=0.8 arg from appstream-builder command line
- specify module_md as module type
- fix saving comps.xml and module_md.yaml into empty copr (with no build)
- module_md.yaml is added to repodata now similarly to appstream.xml
- support for generation of module dist tags
- module_md.yaml uploading for a chroot
- simplified build and action task workflow
- use copy of the mock (chroot) config, not the original in /etc/mock/
* Wed Jun 22 2016 Miroslav Suchý <[email protected]> 1.91-1
- configure more packages to run pylint
- terminate machine which was only partially spawned
- [copr-prune-results] do not sys.exit if prunerepo returns non-zero status,
just raise an exception
- more of log file migration
- claim /var/log/copr-backend in %%files
- adjust log path in runtime files
- update conf file log path directives
- change logdir to /var/log/copr-backend/
* Fri May 27 2016 Miroslav Suchý <[email protected]> 1.90-1
- do not use --log-dir in appstream-builder
* Tue May 24 2016 Miroslav Suchý <[email protected]> 1.89-1
- use correct conditional in requires
* Mon May 23 2016 Miroslav Suchý <[email protected]> 1.88-1
- backend: change logstash requires to soft requires
- 1336360 - allow custom chroots
* Fri May 13 2016 Miroslav Suchý <[email protected]> 1.87-1
- workaround for BZ 1334200
- more info in logs by default
- print seconds just as int
- unsign gpg from forked packages before signing them with new key
- sign forked packages @TODO We need to delsign them first
* Fri May 06 2016 Miroslav Suchý <[email protected]> 1.86-1
- more info in logs by default
- unsign gpg from forked packages before signing them with new key
* Thu May 05 2016 Miroslav Suchý <[email protected]> 1.85-1
- also be tolerant about sign/unsign failures on particular rpm
- just log errors (exception) during particular copr fixing, do not
interrupt the whole process
- added additional check on copr path existence into copr_fix_gpg.py
- allow sudo /usr/bin/rpm for `copr` user
- look into build dirs (subdirs of a chroot) for rpms to be re-signed
- on F24+ use just ansible
- Run rpm-sign with sudo when unsigning
- script to fix gpg keys & rpm signatures
- define functions for deleting gpg signatures from packages
- removed temporary mock workaround from Dockerfile (no
longer needed)
* Thu Apr 28 2016 Miroslav Suchý <[email protected]> 1.84-1
- Bug 1327996 - config_opts['use_host_resolv'] is not set back to
True if it was False before
* Fri Apr 22 2016 Miroslav Suchý <[email protected]> 1.83-1
- run createrepo on forked project (RhBug: 1329076)
- Bug 1327852 - /usr/bin/check_consecutive_build_fails.py errors
- we need to stick to ansible1.9
- more escaping
- prunning down testresults :)
- a few unittests for copr_prune_results.py script
- unit test "fixes"
- fix error when forking into existing project
- (mockremote): improve chroot_setup_cmd replacement for EL-5
- copr_prune_results.py - python path fix
- Bug 1324514 - copr createrepo error messages - fix for errors of
type one
- Bug 1324514 - copr createrepo error messages - fix for errors of
type 2
* Thu Mar 24 2016 Jakub Kadlčík <[email protected]> 1.82-1
- use timeout variable from config
* Mon Mar 14 2016 Jakub Kadlčík <[email protected]> 1.81-1
- support project forking
- support building from PyPI
- support for redis_host, redis_port, redis_db config options
- dockerized-backend project moved under backend/docker
- run createrepo in rawhide_to_release
- specify rawhide name when calling rawhide_to_release
* Fri Jan 29 2016 Miroslav Suchý <[email protected]> 1.80-1
- do not fail when when you receive job with architecture which does not have
queue
- fix 1260780 - Build fails after successful package generation -
just add a log error message pointing to an rsync log
- jobgrabcontrol.py/retask misuse fix
- "localhost-targeted" spawn and terminate playbooks added for testing
- [frontend]implement rawhide to release feature First create new
chroots: python manage.py create_chroot fedora-24-i386 fedora-24-x86_64
- abstraction above [BE <-> JG <-> Builders] channels
- don't traceback backend if frontend is not yet up&running
- do not preserve user and group when rsyncing
* Wed Dec 23 2015 Miroslav Suchý <[email protected]> 1.79-1
- fix packaging issues in epel-7+
* Mon Nov 16 2015 Miroslav Suchý <[email protected]> 1.78-1
- handle_generate_gpg_key skips key creation when signing is disabled
- Added test_handle_generate_gpg_key
- fixed failing tests
- show when createrepo is waiting for lock
- do not block builds when processing too much actions
* Fri Nov 06 2015 Miroslav Suchý <[email protected]> 1.77-1
- we need to have recent python-copr
- create copr-backend-service script to handle all copr services
- [backend] fix not starting job_grab
* Tue Oct 13 2015 Miroslav Suchý <[email protected]> 1.76-1
- createrepo action run infinitely when applied to
deleted project
* Mon Sep 21 2015 Valentin Gologuzov <[email protected]> 1.75-1
- [backend] run copr-backend-log service before other components
* Mon Sep 21 2015 Valentin Gologuzov <[email protected]> 1.74-1
- [backend] add executable bit to run/copr_run_job_grab.py
* Mon Sep 21 2015 Valentin Gologuzov <[email protected]> 1.73-1
- added context manager `local_file_logger`
- eliminated global multiprocessing.Lock
- split backend daemon: extracted RedisLogHandler, JobGrab, VMM
- replace python-bunch with python-munch
- added comps.xml support
* Tue Aug 04 2015 Valentin Gologuzov <[email protected]> 1.72-1
- support new results naming in the build deletion action
- fix BuildJob.results_dir; eliminated MockRemote.pkg_dest_path
- using package name and versiong given in the build task; cleanup;
- handle error's caused by failure to obtain srpm from dist-git
- repairing test for newest changes
- rsync update + several fixes
- building from dist git
- fix vm spawn check: spawner count child processes per build group;
* Wed Jul 01 2015 Valentin Gologuzov <[email protected]> 1.71-1
- add small script to print queues
- AppData supproted
- copy mockchain and rsync logs to resdir (RhBug:1221519)
- note which modules still stops us from migrating to python3
* Mon Jun 15 2015 Miroslav Suchý <[email protected]> 1.70-1
- alter vm_name= regexp
- polishing Bug 1195867 - Move or delete logs when rebuilding failed
build.
- backup only info and log files
- have just one backup directory per results directory
- clean results from previous build
- alter IP= regexp
- disabled appdata until fixed
- unable appdata in createrepo
- more safe VmMaster.check_one_vm_for_dead_builder function
- adding support for AppData
- new requirement form AppData support
- createrepo_unsafe now returns only STDOUT and raise exception on
errors
* Mon Jun 01 2015 Valentin Gologuzov <[email protected]> 1.69-1
- removed creation of symlinks for log.gz
- catch exception during Worker.can_start_job
- config cleanup
* Thu May 28 2015 Valentin Gologuzov <[email protected]> 1.68-1
- [backend] add config option for VM health check timeout
- [backend] moved config parameters from Threshold class into the backend
config file
* Thu May 21 2015 Valentin Gologuzov <[email protected]> 1.67-1
- [backend] Handle unexpected exception VmMaster::check_one_vm_for_dead_builder
* Thu May 21 2015 Valentin Gologuzov <[email protected]> 1.66-1