Remake
remake.cpp
Go to the documentation of this file.
1 /**
2 @mainpage Remake, a build system that bridges the gap between make and redo.
3 
4 As with <b>make</b>, <b>remake</b> uses a centralized rule file, which is
5 named <b>Remakefile</b>. It contains rules with a <em>make</em>-like
6 syntax:
7 
8 @verbatim
9 target1 target2 ... : prerequisite1 prerequisite2 ...
10  shell script
11  that builds
12  the targets
13 @endverbatim
14 
15 A target is known to be up-to-date if all its prerequisites are. If it
16 has no known prerequisites yet the file already exits, it is assumed to
17 be up-to-date. Obsolete targets are rebuilt thanks to the shell script
18 provided by the rule.
19 
20 As with <b>redo</b>, <b>remake</b> supports dynamic dependencies in
21 addition to these static dependencies. Whenever a script executes
22 <tt>remake prerequisite4 prerequisite5 ...</tt>, these prerequisites are
23 rebuilt if they are obsolete. (So <b>remake</b> acts like
24 <b>redo-ifchange</b>.) Moreover, all the dependencies are stored in file
25 <b>.remake</b> so that they are remembered in subsequent runs. Note that
26 dynamic dependencies from previous runs are only used to decide whether a
27 target is obsolete; they are not automatically rebuilt when they are
28 obsolete yet a target depends on them. They will only be rebuilt once the
29 dynamic call to <b>remake</b> is executed.
30 
31 In other words, the following two rules have almost the same behavior.
32 
33 @verbatim
34 target1 target2 ... : prerequisite1 prerequisite2 ...
35  shell script
36 
37 target1 target2 ... :
38  remake prerequisite1 prerequisite2 ...
39  shell script
40 @endverbatim
41 
42 (There is a difference if the targets already exist, have never been
43 built before, and the prerequisites are either younger or obsolete, since
44 the targets will not be rebuilt in the second case.)
45 
46 The above usage of dynamic dependencies is hardly useful. Their strength
47 lies in the fact that they can be computed on the fly:
48 
49 @verbatim
50 %.o : %.c
51  gcc -MMD -MF $@.d -o $@ -c $<
52  remake -r < $@.d
53  rm $@.d
54 
55 %.cmo : %.ml
56  ocamldep $< | remake -r $@
57  ocamlc -c $<
58 
59 after.xml: before.xml rules.xsl
60  xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
61  remake `sed -n -e "\\,//,! s,^.*URL=\"\\([^\"]*\\).*\$,\\1,p" deps`
62  rm deps
63 @endverbatim
64 
65 Note that the first rule fails if any of the header files included by
66 a C source file has to be automatically generated. In that case, one
67 should perform a first call to <b>remake</b> them before calling the
68 compiler. (Dependencies from several calls to <b>remake</b> are
69 cumulative, so they will all be remembered the next time.)
70 
71 \section sec-usage Usage
72 
73 Usage: <tt>remake <i>options</i> <i>targets</i></tt>
74 
75 Options:
76 
77 - <tt>-B</tt>, <tt>--always-make</tt>: Unconditionally make all targets.
78 - <tt>-d</tt>: Echo script commands.
79 - <tt>-f FILE</tt>: Read <tt>FILE</tt> as <b>Remakefile</b>.
80 - <tt>-j[N]</tt>, <tt>--jobs=[N]</tt>: Allow <tt>N</tt> jobs at once;
81  infinite jobs with no argument.
82 - <tt>-k</tt>, <tt>--keep-going</tt>: Keep going when some targets cannot be made.
83 - <tt>-r</tt>: Look up targets from the dependencies on standard input.
84 - <tt>-s</tt>, <tt>--silent</tt>, <tt>--quiet</tt>: Do not echo targets.
85 
86 \section sec-syntax Syntax
87 
88 Lines starting with a space character or a tabulation are assumed to be rule
89 scripts. They are only allowed after a rule header.
90 
91 Lines starting with <tt>#</tt> are considered to be comments and are ignored.
92 They do interrupt rule scripts though.
93 
94 Any other line is either a variable definition or a rule header. If such a
95 line ends with a backslash, the following line break is ignored and the line
96 extends to the next one.
97 
98 Variable definitions are a single name followed by equal followed by a list
99 of names, possibly empty.
100 
101 Rule headers are a nonempty list of names, followed by a colon, followed by
102 another list of names, possibly empty. Basically, the syntax of a rule is as
103 follows:
104 
105 @verbatim
106 targets : prerequisites
107  shell script
108 @endverbatim
109 
110 List of names are space-separated sequences of names. If a name contains a
111 space character, it should be put into double quotes. Names can not be any
112 of the following special characters <tt>:$(),="</tt>. Again, quotation
113 should be used. Quotation marks can be escaped by a backslash inside
114 quoted names.
115 
116 \subsection sec-variables Variables
117 
118 Variables can be used to factor lists of targets or prerequisites. They are
119 expanded as they are encountered during <b>Remakefile</b> parsing.
120 
121 @verbatim
122 VAR2 = a
123 VAR1 = c d
124 VAR2 += $(VAR1) b
125 $(VAR2) e :
126 @endverbatim
127 
128 Variable assignments can appear instead of prerequisites inside non-generic
129 rules with no script. They are then expanded inside the corresponding
130 generic rule.
131 
132 @verbatim
133 foo.o: CFLAGS += -DBAR
134 
135 %.o : %.c
136  gcc $(CFLAGS) -MMD -MF $@.d -o $@ -c $<
137  remake -r < $@.d
138  rm $@.d
139 @endverbatim
140 
141 Note: contrarily to <b>make</b>, variable names have to be enclosed in
142 parentheses. For instance, <tt>$y</tt> is not a shorthand for <tt>\$(y)</tt> and
143 is left unexpanded.
144 
145 \subsection sec-autovars Automatic variables
146 
147 The following special symbols can appear inside scripts:
148 
149 - <tt>$&lt;</tt> expands to the first static prerequisite of the rule.
150 - <tt>$^</tt> expands to all the static prerequisites of the rule, including
151  duplicates if any.
152 - <tt>$\@</tt> expands to the first target of the rule.
153 - <tt>$*</tt> expands to the string that matched <tt>%</tt> in a generic rule.
154 - <tt>$$</tt> expands to a single dollar symbol.
155 
156 Note: contrarily to <b>make</b>, there are no corresponding variables. For
157 instance, <tt>$^</tt> is not a shorthand for <tt>$(^)</tt>. Another
158 difference is that <tt>$\@</tt> is always the first target, not the one that
159 triggered the rule.
160 
161 \subsection sec-functions Built-in functions
162 
163 <b>remake</b> also supports a few built-in functions inspired from <b>make</b>.
164 
165 - <tt>$(addprefix <i>prefix</i>, <i>list</i>)</tt> returns the list obtained
166  by prepending its first argument to each element of its second argument.
167 - <tt>$(addsuffix <i>suffix</i>, <i>list</i>)</tt> returns the list obtained
168  by appending its first argument to each element of its second argument.
169 
170 \subsection sec-order Order-only prerequisites
171 
172 If the static prerequisites of a rule contain a pipe symbol, prerequisites
173 on its right do not cause the targets to become obsolete if they are newer
174 (unless they are also dynamically registered as dependencies). They are
175 meant to be used when the targets do not directly depend on them, but the
176 computation of their dynamic dependencies does.
177 
178 @verbatim
179 %.o : %.c | parser.h
180  gcc -MMD -MF $@.d -o $@ -c $<
181  remake -r < $@.d
182  rm $@.d
183 
184 parser.c parser.h: parser.y
185  yacc -d -o parser.c parser.y
186 @endverbatim
187 
188 \subsection sec-special-tgt Special targets
189 
190 Target <tt>.PHONY</tt> marks its prerequisites as being always obsolete.
191 
192 \subsection sec-special-var Special variables
193 
194 Variable <tt>.OPTIONS</tt> is handled specially. Its content enables some
195 features of <b>remake</b> that are not enabled by default.
196 
197 - <tt>variable-propagation</tt>: When a variable is set in the prerequisite
198  part of a rule, it is propagated to the rules of all the targets this rule
199  depends on. This option also enables variables to be set on the command
200  line. Note that, as in <b>make</b>, this features introduces non-determinism:
201  the content of some variables will depend on the build order.
202 
203 \section sec-semantics Semantics
204 
205 \subsection src-obsolete When are targets obsolete?
206 
207 A target is obsolete:
208 
209 - if there is no file corresponding to the target, or to one of its siblings
210  in a multi-target rule,
211 - if any of its dynamic prerequisites from a previous run or any of its static
212  prerequisites is obsolete,
213 - if the latest file corresponding to its siblings or itself is older than any
214  of its dynamic prerequisites or static prerequisites.
215 
216 In all the other cases, it is assumed to be up-to-date (and so are all its
217 siblings). Note that the last rule above says "latest" and not "earliest". While
218 it might cause some obsolete targets to go unnoticed in corner cases, it allows
219 for the following kind of rules:
220 
221 @verbatim
222 config.h stamp-config_h: config.h.in config.status
223  ./config.status config.h
224  touch stamp-config_h
225 @endverbatim
226 
227 A <tt>config.status</tt> file generally does not update header files (here
228 <tt>config.h</tt>) if they would not change. As a consequence, if not for the
229 <tt>stamp-config_h</tt> file above, a header would always be considered obsolete
230 once one of its prerequisites is modified. Note that touching <tt>config.h</tt>
231 rather than <tt>stamp-config_h</tt> would defeat the point of not updating it
232 in the first place, since the program files would need to be rebuilt.
233 
234 Once all the static prerequisites of a target have been rebuilt, <b>remake</b>
235 checks whether the target still needs to be built. If it was obsolete only
236 because its prerequisites needed to be rebuilt and none of them changed, the
237 target is assumed to be up-to-date.
238 
239 \subsection sec-rules How are targets (re)built?
240 
241 There are two kinds of rules. If any of the targets or prerequisites contains
242 a <tt>%</tt> character, the rule is said to be <em>generic</em>. All the
243 targets of the rule shall then contain a single <tt>%</tt> character. All the
244 other rules are said to be <em>specific</em>.
245 
246 A rule is said to <em>match</em> a given target:
247 
248 - if it is specific and the target appears inside its target list,
249 - if it is generic and there is a way to replace the <tt>%</tt> character
250  from one of its targets so that it matches the given target.
251 
252 When <b>remake</b> tries to build a given target, it looks for a specific rule
253 that matches it. If there is one and its script is nonempty, it uses it to
254 rebuild the target.
255 
256 Otherwise, it looks for a generic rule that matches the target. If there are
257 several matching rules, it chooses the one with the shortest pattern (and if
258 there are several ones, the earliest one). <b>remake</b> then looks for
259 specific rules that match each target of the generic rule. All the
260 prerequisites of these specific rules are added to those of the generic rule.
261 The script of the generic rule is used to build the target.
262 
263 Example:
264 
265 @verbatim
266 t%1 t2%: p1 p%2
267  commands building t%1 and t2%
268 
269 t2z: p4
270  commands building t2z
271 
272 ty1: p3
273 
274 # t2x is built by the first rule (which also builds tx1) and its prerequisites are p1, px2
275 # t2y is built by the first rule (which also builds ty1) and its prerequisites are p1, py2, p3
276 # t2z is built by the second rule and its prerequisite is p4
277 @endverbatim
278 
279 The set of rules from <b>Remakefile</b> is ill-formed:
280 
281 - if any specific rule matching a target of the generic rule has a nonempty script,
282 - if any target of the generic rule is matched by a generic rule with a shorter pattern.
283 
284 \section sec-compilation Compilation
285 
286 - On Linux, MacOSX, and BSD: <tt>g++ -o remake remake.cpp</tt>
287 - On Windows: <tt>g++ -o remake.exe remake.cpp -lws2_32</tt>
288 
289 Installing <b>remake</b> is needed only if <b>Remakefile</b> does not
290 specify the path to the executable for its recursive calls. Thanks to its
291 single source file, <b>remake</b> can be shipped inside other packages and
292 built at configuration time.
293 
294 \section sec-differences Differences with other build systems
295 
296 Differences with <b>make</b>:
297 
298 - Dynamic dependencies are supported.
299 - For rules with multiple targets, the shell script is executed only once
300  and is assumed to build all the targets. There is no need for
301  convoluted rules that are robust enough for parallel builds. For generic
302  rules, this is similar to the behavior of pattern rules from <b>gmake</b>.
303 - As with <b>redo</b>, only one shell is run when executing a script,
304  rather than one per script line. Note that the shells are run with
305  option <tt>-e</tt>, thus causing them to exit as soon as an error is
306  encountered.
307 - The prerequisites of generic rules (known as implicit rules in make lingo)
308  are not used to decide between several of them. <b>remake</b> does not
309  select one for which it could satisfy the dependencies.
310 - Variables and built-in functions are expanded as they are encountered
311  during <b>Remakefile</b> parsing.
312 - Target-specific variables are not propagated, unless specifically enabled,
313  since this causes non-deterministic builds. This is the same for variables
314  set on the command line.
315 
316 Differences with <b>redo</b>:
317 
318 - As with <b>make</b>, it is possible to write the following kind of rules
319  in <b>remake</b>.
320 @verbatim
321 Remakefile: Remakefile.in ./config.status
322  ./config.status Remakefile
323 @endverbatim
324 - If a target is already built the first time <b>remake</b> runs, it still
325  uses the static prerequisites of rules mentioning it to check whether it
326  needs to be rebuilt. It does not assume it to be up-to-date. As with
327  <b>redo</b> though, if its obsolete status would be due to a dynamic
328  prerequisite, it will go unnoticed; it should be removed beforehand.
329 - Multiple targets are supported.
330 - <b>remake</b> has almost no features: no checksum-based dependencies, no
331  compatibility with job servers, etc.
332 
333 \section sec-limitations Limitations
334 
335 - If a rule script calls <b>remake</b>, the current working directory should
336  be the directory containing <b>Remakefile</b> (or the working directory
337  from the original <b>remake</b> if it was called with option <b>-f</b>).
338 - As with <b>make</b>, variables passed on the command line should keep
339  the same values, to ensure deterministic builds.
340 - Some cases of ill-formed rules are not caught by <b>remake</b> and can
341  thus lead to unpredictable behaviors.
342 
343 \section sec-links Links
344 
345 @see http://cr.yp.to/redo.html for the philosophy of <b>redo</b> and
346 https://github.com/apenwarr/redo for an implementation and some comprehensive documentation.
347 
348 \section sec-licensing Licensing
349 
350 @author Guillaume Melquiond
351 @version 0.12
352 @date 2012-2014
353 @copyright
354 This program is free software: you can redistribute it and/or modify
355 it under the terms of the GNU General Public License as published by
356 the Free Software Foundation, either version 3 of the License, or
357 (at your option) any later version.
358 \n
359 This program is distributed in the hope that it will be useful,
360 but WITHOUT ANY WARRANTY; without even the implied warranty of
361 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
362 GNU General Public License for more details.
363 
364 \section sec-internals Internals
365 
366 The parent <b>remake</b> process acts as a server. The other ones have a
367 REMAKE_SOCKET environment variable that tells them how to contact the
368 server. They send the content of the REMAKE_JOB_ID environment variable,
369 so that the server can associate the child targets to the jobs that
370 spawned them. They then wait for completion and exit with the status
371 returned by the server. This is handled by #client_mode.
372 
373 The server calls #load_dependencies and #save_dependencies to serialize
374 dynamic dependencies from <b>.remake</b>. It loads <b>Remakefile</b> with
375 #load_rules. It then runs #server_mode, which calls #server_loop.
376 
377 When building a target, the following sequence of events happens:
378 
379 - #start calls #find_rule (and #find_generic_rule) to get the rule.
380 - It then creates a pseudo-client if the rule has static dependencies, or
381  calls #run_script otherwise. In both cases, a new job is created; the
382  rule and the variables are stored into #jobs.
383 - #run_script creates a shell process and stores it in #job_pids. It
384  increases #running_jobs.
385 - The child process possibly calls <b>remake</b> with a list of targets.
386 - #accept_client receives a build request from a child process and adds
387  it to #clients. It also records the new dependencies of the job into
388  #dependencies. It increases #waiting_jobs.
389 - #handle_clients uses #get_status to look up the obsoleteness of the
390  targets.
391 - Once the targets of a request have been built or one of them has failed,
392  #handle_clients calls #complete_request and removes the request from
393  #clients.
394 - If the build targets come from a pseudo-client, #complete_request calls
395  #run_script. Otherwise it sends the reply to the corresponding child
396  process and decreases #waiting_jobs.
397 - When a child process ends, #server_loop calls #finalize_job, which
398  removes the process from #job_pids, decreases #running_jobs, and calls
399  #complete_job.
400 - #complete_job removes the job from #jobs and calls #update_status
401  to change the status of the targets. It also removes the target files in
402  case of failure.
403 */
404 
405 #ifdef _WIN32
406 #define WIN32_LEAN_AND_MEAN
407 #define WINDOWS
408 #endif
409 
410 #include <fstream>
411 #include <iostream>
412 #include <list>
413 #include <map>
414 #include <set>
415 #include <sstream>
416 #include <string>
417 #include <vector>
418 #include <cassert>
419 #include <cstdlib>
420 #include <cstring>
421 #include <ctime>
422 #include <errno.h>
423 #include <fcntl.h>
424 #include <signal.h>
425 #include <unistd.h>
426 #include <sys/stat.h>
427 #include <sys/types.h>
428 
429 #ifdef __APPLE__
430 #define MACOSX
431 #endif
432 
433 #ifdef __linux__
434 #define LINUX
435 #endif
436 
437 #ifdef WINDOWS
438 #include <windows.h>
439 #include <winbase.h>
440 #include <winsock2.h>
441 #define pid_t HANDLE
442 typedef SOCKET socket_t;
443 #else
444 #include <sys/socket.h>
445 #include <sys/un.h>
446 #include <sys/wait.h>
447 typedef int socket_t;
448 enum { INVALID_SOCKET = -1 };
449 extern char **environ;
450 #endif
451 
452 #if defined(WINDOWS) || defined(MACOSX)
453 enum { MSG_NOSIGNAL = 0 };
454 #endif
455 
456 typedef std::list<std::string> string_list;
457 
458 typedef std::set<std::string> string_set;
459 
460 /**
461  * Reference-counted shared object.
462  * @note The default constructor delays the creation of the object until it
463  * is first dereferenced.
464  */
465 template<class T>
466 struct ref_ptr
467 {
468  struct content
469  {
470  size_t cnt;
471  T val;
472  content(): cnt(1) {}
473  content(T const &t): cnt(1), val(t) {}
474  };
475  mutable content *ptr;
476  ref_ptr(): ptr(NULL) {}
477  ref_ptr(T const &t): ptr(new content(t)) {}
478  ref_ptr(ref_ptr const &p): ptr(p.ptr) { if (ptr) ++ptr->cnt; }
479  ~ref_ptr() { if (ptr && --ptr->cnt == 0) delete ptr; }
481  {
482  if (ptr == p.ptr) return *this;
483  if (ptr && --ptr->cnt == 0) delete ptr;
484  ptr = p.ptr;
485  if (ptr) ++ptr->cnt;
486  return *this;
487  }
488  T &operator*() const
489  {
490  if (!ptr) ptr = new content;
491  return ptr->val;
492  }
493  T *operator->() const { return &**this; }
494 };
495 
497 {
500 };
501 
502 typedef std::map<std::string, ref_ptr<dependency_t> > dependency_map;
503 
504 typedef std::map<std::string, string_list> variable_map;
505 
506 /**
507  * Build status of a target.
508  */
510 {
511  Uptodate, ///< Target is up-to-date.
512  Todo, ///< Target is missing or obsolete.
513  Recheck, ///< Target has an obsolete dependency.
514  Running, ///< Target is being rebuilt.
515  RunningRecheck, ///< Static prerequisites are being rebuilt.
516  Remade, ///< Target was successfully rebuilt.
517  Failed ///< Build failed for target.
518 };
519 
520 /**
521  * Build status of a target.
522  */
523 struct status_t
524 {
525  status_e status; ///< Actual status.
526  time_t last; ///< Last-modified date.
527 };
528 
529 typedef std::map<std::string, status_t> status_map;
530 
531 /**
532  * Delayed assignment to a variable.
533  */
534 struct assign_t
535 {
536  bool append;
538 };
539 
540 typedef std::map<std::string, assign_t> assign_map;
541 
542 /**
543  * A rule loaded from Remakefile.
544  */
545 struct rule_t
546 {
547  string_list targets; ///< Files produced by this rule.
548  string_list deps; ///< Dependencies used for an implicit call to remake at the start of the script.
549  string_list wdeps; ///< Like #deps, except that they are not registered as dependencies.
550  assign_map assigns; ///< Assignment of variables.
551  std::string script; ///< Shell script for building the targets.
552 };
553 
554 typedef std::list<rule_t> rule_list;
555 
556 typedef std::map<std::string, ref_ptr<rule_t> > rule_map;
557 
558 /**
559  * A job created from a set of rules.
560  */
561 
562 struct job_t
563 {
564  rule_t rule; ///< Original rule.
565  std::string stem; ///< Pattern used to instantiate the generic rule, if any.
566  variable_map vars; ///< Values of local variables.
567 };
568 
569 typedef std::map<int, job_t> job_map;
570 
571 typedef std::map<pid_t, int> pid_job_map;
572 
573 /**
574  * Client waiting for a request to complete.
575  *
576  * There are two kinds of clients:
577  * - real clients, which are instances of remake created by built scripts,
578  * - pseudo clients, which are created by the server to build specific targets.
579  *
580  * Among pseudo clients, there are two categories:
581  * - original clients, which are created for the targets passed on the
582  * command line by the user or for the initial regeneration of the rule file,
583  * - dependency clients, which are created to handle rules that have
584  * explicit dependencies and thus to emulate a call to remake.
585  */
586 struct client_t
587 {
588  socket_t socket; ///< Socket used to reply to the client (invalid for pseudo clients).
589  int job_id; ///< Job for which the built script called remake and spawned the client (negative for original clients).
590  bool failed; ///< Whether some targets failed in mode -k.
591  string_list pending; ///< Targets not yet started.
592  string_set running; ///< Targets being built.
593  variable_map vars; ///< Variables set on request.
594  bool delayed; ///< Whether it is a dependency client and a script has to be started on request completion.
595  client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(false) {}
596 };
597 
598 typedef std::list<client_t> client_list;
599 
600 /**
601  * Map from variable names to their content.
602  * Initialized with the values passed on the command line.
603  */
605 
606 /**
607  * Map from targets to their known dependencies.
608  */
610 
611 /**
612  * Map from targets to their build status.
613  */
615 
616 /**
617  * Set of generic rules loaded from Remakefile.
618  */
620 
621 /**
622  * Map from targets to specific rules loaded from Remakefile.
623  */
625 
626 /**
627  * Map of jobs being built.
628  */
629 static job_map jobs;
630 
631 /**
632  * Map from jobs to shell pids.
633  */
635 
636 /**
637  * List of clients waiting for a request to complete.
638  * New clients are put to front, so that the build process is depth-first.
639  */
641 
642 /**
643  * Maximum number of parallel jobs (non-positive if unbounded).
644  * Can be modified by the -j option.
645  */
646 static int max_active_jobs = 1;
647 
648 /**
649  * Whether to keep building targets in case of failure.
650  * Can be modified by the -k option.
651  */
652 static bool keep_going = false;
653 
654 /**
655  * Number of jobs currently running:
656  * - it increases when a process is created in #run_script,
657  * - it decreases when a completion message is received in #finalize_job.
658  *
659  * @note There might be some jobs running while #clients is empty.
660  * Indeed, if a client requested two targets to be rebuilt, if they
661  * are running concurrently, if one of them fails, the client will
662  * get a failure notice and might terminate before the other target
663  * finishes.
664  */
665 static int running_jobs = 0;
666 
667 /**
668  * Number of jobs currently waiting for a build request to finish:
669  * - it increases when a build request is received in #accept_client
670  * (since the client is presumably waiting for the reply),
671  * - it decreases when a reply is sent in #complete_request.
672  */
673 static int waiting_jobs = 0;
674 
675 /**
676  * Global counter used to produce increasing job numbers.
677  * @see jobs
678  */
679 static int job_counter = 0;
680 
681 /**
682  * Socket on which the server listens for client request.
683  */
685 
686 /**
687  * Whether the request of an original client failed.
688  */
689 static bool build_failure;
690 
691 #ifndef WINDOWS
692 /**
693  * Name of the server socket in the file system.
694  */
695 static char *socket_name;
696 #endif
697 
698 /**
699  * Name of the first target of the first specific rule, used for default run.
700  */
701 static std::string first_target;
702 
703 /**
704  * Whether a short message should be displayed for each target.
705  */
706 static bool show_targets = true;
707 
708 /**
709  * Whether script commands are echoed.
710  */
711 static bool echo_scripts = false;
712 
713 /**
714  * Time at the start of the program.
715  */
716 static time_t now = time(NULL);
717 
718 /**
719  * Directory with respect to which command-line names are relative.
720  */
721 static std::string working_dir;
722 
723 /**
724  * Directory with respect to which targets are relative.
725  */
726 static std::string prefix_dir;
727 
728 /**
729  * Whether the prefix directory is different from #working_dir.
730  */
731 static bool changed_prefix_dir;
732 
733 /**
734  * Whether target-specific variables are propagated to prerequisites.
735  */
736 static bool propagate_vars = false;
737 
738 /**
739  * Whether targets are unconditionally obsolete.
740  */
741 static bool obsolete_targets = false;
742 
743 #ifndef WINDOWS
744 static volatile sig_atomic_t got_SIGCHLD = 0;
745 
746 static void sigchld_handler(int)
747 {
748  got_SIGCHLD = 1;
749 }
750 
751 static void sigint_handler(int)
752 {
753  // Child processes will receive the signal too, so just prevent
754  // new jobs from starting and wait for the running jobs to fail.
755  keep_going = false;
756 }
757 #endif
758 
759 struct log
760 {
761  bool active, open;
762  int depth;
763  log(): active(false), open(false), depth(0)
764  {
765  }
766  std::ostream &operator()()
767  {
768  if (open) std::cerr << std::endl;
769  assert(depth >= 0);
770  std::cerr << std::string(depth * 2, ' ');
771  open = false;
772  return std::cerr;
773  }
774  std::ostream &operator()(bool o)
775  {
776  if (o && open) std::cerr << std::endl;
777  if (!o) --depth;
778  assert(depth >= 0);
779  if (o || !open) std::cerr << std::string(depth * 2, ' ');
780  if (o) ++depth;
781  open = o;
782  return std::cerr;
783  }
784 };
785 
786 static log debug;
787 
789 {
792  {
793  }
795  {
796  if (debug.active && still_open) debug(false) << "done\n";
797  }
798 };
799 
800 #define DEBUG if (debug.active) debug()
801 #define DEBUG_open log_auto_close auto_close; if (debug.active) debug(true)
802 #define DEBUG_close if ((auto_close.still_open = false), debug.active) debug(false)
803 
804 /**
805  * Strong typedef for strings that need escaping.
806  * @note The string is stored as a reference, so the constructed object is
807  * meant to be immediately consumed.
808  */
810 {
811  std::string const &input;
812  escape_string(std::string const &s): input(s) {}
813 };
814 
815 /**
816  * Write the string in @a se to @a out if it does not contain any special
817  * characters, a quoted and escaped string otherwise.
818  */
819 static std::ostream &operator<<(std::ostream &out, escape_string const &se)
820 {
821  std::string const &s = se.input;
822  char const *quoted_char = ",: '";
823  char const *escaped_char = "\"\\$!";
824  bool need_quotes = false;
825  char *buf = NULL;
826  size_t len = s.length(), last = 0, j = 0;
827  for (size_t i = 0; i < len; ++i)
828  {
829  if (strchr(escaped_char, s[i]))
830  {
831  need_quotes = true;
832  if (!buf) buf = new char[len * 2];
833  memcpy(&buf[j], &s[last], i - last);
834  j += i - last;
835  buf[j++] = '\\';
836  buf[j++] = s[i];
837  last = i + 1;
838  }
839  if (!need_quotes && strchr(quoted_char, s[i]))
840  need_quotes = true;
841  }
842  if (!need_quotes) return out << s;
843  out << '"';
844  if (!buf) return out << s << '"';
845  out.write(buf, j);
846  out.write(&s[last], len - last);
847  delete[] buf;
848  return out << '"';
849 }
850 
851 /**
852  * @defgroup paths Path helpers
853  *
854  * @{
855  */
856 
857 /**
858  * Initialize #working_dir.
859  */
860 static void init_working_dir()
861 {
862  char buf[1024];
863  char *res = getcwd(buf, sizeof(buf));
864  if (!res)
865  {
866  perror("Failed to get working directory");
867  exit(EXIT_FAILURE);
868  }
869  working_dir = buf;
870 #ifdef WINDOWS
871  for (size_t i = 0, l = working_dir.size(); i != l; ++i)
872  {
873  if (working_dir[i] == '\\') working_dir[i] = '/';
874  }
875 #endif
877 }
878 
879 /**
880  * Initialize #prefix_dir and switch to it.
881  */
882 static void init_prefix_dir()
883 {
884  for (;;)
885  {
886  struct stat s;
887  if (stat((prefix_dir + "/Remakefile").c_str(), &s) == 0)
888  {
889  if (!changed_prefix_dir) return;
890  if (chdir(prefix_dir.c_str()))
891  {
892  perror("Failed to change working directory");
893  exit(EXIT_FAILURE);
894  }
895  if (show_targets)
896  {
897  std::cout << "remake: Entering directory `" << prefix_dir << '\'' << std::endl;
898  }
899  return;
900  }
901  size_t pos = prefix_dir.find_last_of('/');
902  if (pos == std::string::npos)
903  {
904  std::cerr << "Failed to locate Remakefile in the current directory or one of its parents" << std::endl;
905  exit(EXIT_FAILURE);
906  }
907  prefix_dir.erase(pos);
908  changed_prefix_dir = true;
909  }
910 }
911 
912 /**
913  * Normalize an absolute path with respect to @a p.
914  * Paths outside the subtree are left unchanged.
915  */
916 static std::string normalize_abs(std::string const &s, std::string const &p)
917 {
918  size_t l = p.length();
919  if (s.compare(0, l, p)) return s;
920  size_t ll = s.length();
921  if (ll == l) return ".";
922  if (s[l] != '/')
923  {
924  size_t pos = s.rfind('/', l);
925  assert(pos != std::string::npos);
926  return s.substr(pos + 1);
927  }
928  if (ll == l + 1) return ".";
929  return s.substr(l + 1);
930 }
931 
932 /**
933  * Normalize path @a s (possibly relative to @a w) with respect to @a p.
934  *
935  * - If both @a p and @a w are empty, the function just removes ".", "..", "//".
936  * - If only @a p is empty, the function returns an absolute path.
937  */
938 static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
939 {
940 #ifdef WINDOWS
941  char const *delim = "/\\";
942 #else
943  char delim = '/';
944 #endif
945  size_t pos = s.find_first_of(delim);
946  if (pos == std::string::npos && w == p) return s;
947  bool absolute = pos == 0;
948  if (!absolute && w != p && !w.empty())
949  return normalize(w + '/' + s, w, p);
950  size_t prev = 0, len = s.length();
951  string_list l;
952  for (;;)
953  {
954  if (pos != prev)
955  {
956  std::string n = s.substr(prev, pos - prev);
957  if (n == "..")
958  {
959  if (!l.empty()) l.pop_back();
960  else if (!absolute && !w.empty())
961  return normalize(w + '/' + s, w, p);
962  }
963  else if (n != ".")
964  l.push_back(n);
965  }
966  ++pos;
967  if (pos >= len) break;
968  prev = pos;
969  pos = s.find_first_of(delim, prev);
970  if (pos == std::string::npos) pos = len;
971  }
972  string_list::const_iterator i = l.begin(), i_end = l.end();
973  if (i == i_end) return absolute ? "/" : ".";
974  std::string n;
975  if (absolute) n.push_back('/');
976  n.append(*i);
977  for (++i; i != i_end; ++i)
978  {
979  n.push_back('/');
980  n.append(*i);
981  }
982  if (absolute && !p.empty()) return normalize_abs(n, p);
983  return n;
984 }
985 
986 /**
987  * Normalize the content of a list of targets.
988  */
989 static void normalize_list(string_list &l, std::string const &w, std::string const &p)
990 {
991  for (string_list::iterator i = l.begin(),
992  i_end = l.end(); i != i_end; ++i)
993  {
994  *i = normalize(*i, w, p);
995  }
996 }
997 
998 /** @} */
999 
1000 /**
1001  * @defgroup lexer Lexer
1002  *
1003  * @{
1004  */
1005 
1006 /**
1007  * Skip spaces.
1008  */
1009 static void skip_spaces(std::istream &in)
1010 {
1011  char c;
1012  while (strchr(" \t", (c = in.get()))) {}
1013  if (in.good()) in.putback(c);
1014 }
1015 
1016 /**
1017  * Skip empty lines.
1018  */
1019 static void skip_empty(std::istream &in)
1020 {
1021  char c;
1022  while (strchr("\r\n", (c = in.get()))) {}
1023  if (in.good()) in.putback(c);
1024 }
1025 
1026 /**
1027  * Skip end of line. If @a multi is true, skip the following empty lines too.
1028  * @return true if there was a line to end.
1029  */
1030 static bool skip_eol(std::istream &in, bool multi = false)
1031 {
1032  char c = in.get();
1033  if (c == '\r') c = in.get();
1034  if (c != '\n' && in.good()) in.putback(c);
1035  if (c != '\n' && !in.eof()) return false;
1036  if (multi) skip_empty(in);
1037  return true;
1038 }
1039 
1040 enum
1041 {
1043  Word = 1 << 1,
1044  Colon = 1 << 2,
1045  Equal = 1 << 3,
1046  Dollarpar = 1 << 4,
1047  Rightpar = 1 << 5,
1048  Comma = 1 << 6,
1049  Plusequal = 1 << 7,
1050  Pipe = 1 << 8,
1051 };
1052 
1053 /**
1054  * Skip spaces and peek at the next token.
1055  * If it is one of @a mask, skip it (if it is not Word) and return it.
1056  * @note For composite tokens allowed by @a mask, input characters might
1057  * have been eaten even for an Unexpected result.
1058  */
1059 static int expect_token(std::istream &in, int mask)
1060 {
1061  while (true)
1062  {
1063  skip_spaces(in);
1064  char c = in.peek();
1065  if (!in.good()) return Unexpected;
1066  int tok;
1067  switch (c)
1068  {
1069  case '\r':
1070  case '\n': return Unexpected;
1071  case ':': tok = Colon; break;
1072  case ',': tok = Comma; break;
1073  case '=': tok = Equal; break;
1074  case ')': tok = Rightpar; break;
1075  case '|': tok = Pipe; break;
1076  case '$':
1077  if (!(mask & Dollarpar)) return Unexpected;
1078  in.ignore(1);
1079  tok = Dollarpar;
1080  if (in.peek() != '(') return Unexpected;
1081  break;
1082  case '+':
1083  if (!(mask & Plusequal)) return Unexpected;
1084  in.ignore(1);
1085  tok = Plusequal;
1086  if (in.peek() != '=') return Unexpected;
1087  break;
1088  case '\\':
1089  in.ignore(1);
1090  if (skip_eol(in)) continue;
1091  in.putback('\\');
1092  return mask & Word ? Word : Unexpected;
1093  default:
1094  return mask & Word ? Word : Unexpected;
1095  }
1096  if (!(tok & mask)) return Unexpected;
1097  in.ignore(1);
1098  return tok;
1099  }
1100 }
1101 
1102 /**
1103  * Read a (possibly quoted) word.
1104  */
1105 static std::string read_word(std::istream &in, bool detect_equal = true)
1106 {
1107  int c = in.peek();
1108  std::string res;
1109  if (!in.good()) return res;
1110  char const *separators = " \t\r\n$(),:";
1111  bool quoted = c == '"';
1112  if (quoted) in.ignore(1);
1113  bool plus = false;
1114  while (true)
1115  {
1116  c = in.peek();
1117  if (!in.good()) return res;
1118  if (quoted)
1119  {
1120  in.ignore(1);
1121  if (c == '\\')
1122  res += in.get();
1123  else if (c == '"')
1124  quoted = false;
1125  else
1126  res += c;
1127  continue;
1128  }
1129  if (detect_equal && c == '=')
1130  {
1131  if (plus) in.putback('+');
1132  return res;
1133  }
1134  if (plus)
1135  {
1136  res += '+';
1137  plus = false;
1138  }
1139  if (strchr(separators, c)) return res;
1140  in.ignore(1);
1141  if (detect_equal && c == '+') plus = true;
1142  else res += c;
1143  }
1144 }
1145 
1146 /** @} */
1147 
1148 /**
1149  * @defgroup stream Token streams
1150  *
1151  * @{
1152  */
1153 
1154 /**
1155  * Possible results from word producers.
1156  */
1158 {
1162 };
1163 
1164 /**
1165  * Interface for word producers.
1166  */
1168 {
1169  virtual ~generator() {}
1170  virtual input_status next(std::string &) = 0;
1171 };
1172 
1173 /**
1174  * Generator for the words of a variable.
1175  */
1177 {
1178  std::string name;
1179  string_list::const_iterator vcur, vend;
1180  variable_generator(std::string const &, variable_map const *);
1181  input_status next(std::string &);
1182 };
1183 
1185  variable_map const *local_variables): name(n)
1186 {
1187  if (local_variables)
1188  {
1189  variable_map::const_iterator i = local_variables->find(name);
1190  if (i != local_variables->end())
1191  {
1192  vcur = i->second.begin();
1193  vend = i->second.end();
1194  return;
1195  }
1196  }
1197  variable_map::const_iterator i = variables.find(name);
1198  if (i == variables.end()) return;
1199  vcur = i->second.begin();
1200  vend = i->second.end();
1201 }
1202 
1204 {
1205  if (vcur != vend)
1206  {
1207  res = *vcur;
1208  ++vcur;
1209  return Success;
1210  }
1211  return Eof;
1212 }
1213 
1214 /**
1215  * Generator for the words of an input stream.
1216  */
1218 {
1219  std::istream &in;
1223  input_generator(std::istream &i, variable_map const *lv, bool e = false)
1224  : in(i), nested(NULL), local_variables(lv), earliest_exit(e), done(false) {}
1225  input_status next(std::string &);
1226  ~input_generator() { assert(!nested); }
1227 };
1228 
1229 static generator *get_function(input_generator const &, std::string const &);
1230 
1232 {
1233  if (nested)
1234  {
1235  restart:
1236  input_status s = nested->next(res);
1237  if (s == Success) return Success;
1238  delete nested;
1239  nested = NULL;
1240  if (s == SyntaxError) return SyntaxError;
1241  }
1242  if (done) return Eof;
1243  if (earliest_exit) done = true;
1244  switch (expect_token(in, Word | Dollarpar))
1245  {
1246  case Word:
1247  res = read_word(in, false);
1248  return Success;
1249  case Dollarpar:
1250  {
1251  std::string name = read_word(in, false);
1252  if (name.empty()) return SyntaxError;
1253  if (expect_token(in, Rightpar))
1255  else
1256  {
1257  nested = get_function(*this, name);
1258  if (!nested) return SyntaxError;
1259  }
1260  goto restart;
1261  }
1262  default:
1263  return Eof;
1264  }
1265 }
1266 
1267 /**
1268  * Read a list of words from an input generator.
1269  * @return false if a syntax error was encountered.
1270  */
1272 {
1273  while (true)
1274  {
1275  res.push_back(std::string());
1276  input_status s = in.next(res.back());
1277  if (s == Success) continue;
1278  res.pop_back();
1279  return s == Eof;
1280  }
1281 }
1282 
1283 static bool read_words(std::istream &in, string_list &res)
1284 {
1285  input_generator gen(in, NULL);
1286  return read_words(gen, res);
1287 }
1288 
1289 /**
1290  * Generator for the result of function addprefix.
1291  */
1293 {
1296  string_list::const_iterator prei;
1297  size_t prej, prel;
1298  std::string suf;
1299  addprefix_generator(input_generator const &, bool &);
1300  input_status next(std::string &);
1301 };
1302 
1304  : gen(top.in, top.local_variables)
1305 {
1306  if (!read_words(gen, pre)) return;
1307  if (!expect_token(gen.in, Comma)) return;
1308  prej = 0;
1309  prel = pre.size();
1310  ok = true;
1311 }
1312 
1314 {
1315  if (prej)
1316  {
1317  produce:
1318  if (prej == prel)
1319  {
1320  res = *prei + suf;
1321  prej = 0;
1322  }
1323  else
1324  {
1325  res = *prei++;
1326  ++prej;
1327  }
1328  return Success;
1329  }
1330  switch (gen.next(res))
1331  {
1332  case Success:
1333  if (!prel) return Success;
1334  prei = pre.begin();
1335  prej = 1;
1336  suf = res;
1337  goto produce;
1338  case Eof:
1339  return expect_token(gen.in, Rightpar) ? Eof : SyntaxError;
1340  default:
1341  return SyntaxError;
1342  }
1343 }
1344 
1345 /**
1346  * Generator for the result of function addsuffix.
1347  */
1349 {
1352  string_list::const_iterator sufi;
1353  size_t sufj, sufl;
1354  std::string pre;
1355  addsuffix_generator(input_generator const &, bool &);
1356  input_status next(std::string &);
1357 };
1358 
1360  : gen(top.in, top.local_variables)
1361 {
1362  if (!read_words(gen, suf)) return;
1363  if (!expect_token(gen.in, Comma)) return;
1364  sufj = 0;
1365  sufl = suf.size();
1366  ok = true;
1367 }
1368 
1370 {
1371  if (sufj)
1372  {
1373  if (sufj != sufl)
1374  {
1375  res = *sufi++;
1376  ++sufj;
1377  return Success;
1378  }
1379  sufj = 0;
1380  }
1381  switch (gen.next(res))
1382  {
1383  case Success:
1384  if (!sufl) return Success;
1385  sufi = suf.begin();
1386  sufj = 1;
1387  res += *sufi++;
1388  return Success;
1389  case Eof:
1390  return expect_token(gen.in, Rightpar) ? Eof : SyntaxError;
1391  default:
1392  return SyntaxError;
1393  }
1394 }
1395 
1396 /**
1397  * Return a generator for function @a name.
1398  */
1399 static generator *get_function(input_generator const &in, std::string const &name)
1400 {
1401  skip_spaces(in.in);
1402  generator *g = NULL;
1403  bool ok = false;
1404  if (name == "addprefix") g = new addprefix_generator(in, ok);
1405  else if (name == "addsuffix") g = new addsuffix_generator(in, ok);
1406  if (!g || ok) return g;
1407  delete g;
1408  return NULL;
1409 }
1410 
1411 /** @} */
1412 
1413 /**
1414  * @defgroup database Dependency database
1415  *
1416  * @{
1417  */
1418 
1419 /**
1420  * Load dependencies from @a in.
1421  */
1422 static void load_dependencies(std::istream &in)
1423 {
1424  if (false)
1425  {
1426  error:
1427  std::cerr << "Failed to load database" << std::endl;
1428  exit(EXIT_FAILURE);
1429  }
1430 
1431  while (!in.eof())
1432  {
1433  string_list targets;
1434  if (!read_words(in, targets)) goto error;
1435  if (in.eof()) return;
1436  if (targets.empty()) goto error;
1437  DEBUG << "reading dependencies of target " << targets.front() << std::endl;
1438  if (in.get() != ':') goto error;
1440  dep->targets = targets;
1441  string_list deps;
1442  if (!read_words(in, deps)) goto error;
1443  dep->deps.insert(deps.begin(), deps.end());
1444  for (string_list::const_iterator i = targets.begin(),
1445  i_end = targets.end(); i != i_end; ++i)
1446  {
1447  dependencies[*i] = dep;
1448  }
1449  skip_empty(in);
1450  }
1451 }
1452 
1453 /**
1454  * Load known dependencies from file <tt>.remake</tt>.
1455  */
1456 static void load_dependencies()
1457 {
1458  DEBUG_open << "Loading database... ";
1459  std::ifstream in(".remake");
1460  if (!in.good())
1461  {
1462  DEBUG_close << "not found\n";
1463  return;
1464  }
1465  load_dependencies(in);
1466 }
1467 
1468 
1469 /**
1470  * Save all the dependencies in file <tt>.remake</tt>.
1471  */
1472 static void save_dependencies()
1473 {
1474  DEBUG_open << "Saving database... ";
1475  std::ofstream db(".remake");
1476  while (!dependencies.empty())
1477  {
1478  ref_ptr<dependency_t> dep = dependencies.begin()->second;
1479  for (string_list::const_iterator i = dep->targets.begin(),
1480  i_end = dep->targets.end(); i != i_end; ++i)
1481  {
1482  db << escape_string(*i) << ' ';
1483  dependencies.erase(*i);
1484  }
1485  db << ':';
1486  for (string_set::const_iterator i = dep->deps.begin(),
1487  i_end = dep->deps.end(); i != i_end; ++i)
1488  {
1489  db << ' ' << escape_string(*i);
1490  }
1491  db << std::endl;
1492  }
1493 }
1494 
1495 /** @} */
1496 
1497 static void merge_rule(rule_t &dest, rule_t const &src);
1498 
1499 /**
1500  * @defgroup parser Rule parser
1501  *
1502  * @{
1503  */
1504 
1505 /**
1506  * Register a specific rule with an empty script:
1507  *
1508  * - Check that none of the targets already has an associated rule with a
1509  * nonempty script.
1510  * - Create a new rule with a single target for each target, if needed.
1511  * - Add the prerequisites of @a rule to all these associated rules.
1512  */
1513 static void register_transparent_rule(rule_t const &rule, string_list const &targets)
1514 {
1515  assert(rule.script.empty());
1516  for (string_list::const_iterator i = targets.begin(),
1517  i_end = targets.end(); i != i_end; ++i)
1518  {
1519  std::pair<rule_map::iterator, bool> j =
1520  specific_rules.insert(std::make_pair(*i, ref_ptr<rule_t>()));
1521  ref_ptr<rule_t> &r = j.first->second;
1522  if (j.second)
1523  {
1524  r = ref_ptr<rule_t>(rule);
1525  r->targets = string_list(1, *i);
1526  continue;
1527  }
1528  if (!r->script.empty())
1529  {
1530  std::cerr << "Failed to load rules: " << *i
1531  << " cannot be the target of several rules" << std::endl;
1532  exit(EXIT_FAILURE);
1533  }
1534  assert(r->targets.size() == 1 && r->targets.front() == *i);
1535  merge_rule(*r, rule);
1536  }
1537 
1538  for (string_list::const_iterator i = targets.begin(),
1539  i_end = targets.end(); i != i_end; ++i)
1540  {
1542  if (dep->targets.empty()) dep->targets.push_back(*i);
1543  dep->deps.insert(rule.deps.begin(), rule.deps.end());
1544  }
1545 }
1546 
1547 /**
1548  * Register a specific rule with a nonempty script:
1549  *
1550  * - Check that none of the targets already has an associated rule.
1551  * - Create a single shared rule and associate it to all the targets.
1552  * - Merge the prerequisites of all the targets into a single set and
1553  * add the prerequisites of the rule to it. (The preexisting
1554  * prerequisites, if any, come from a previous run.)
1555  */
1556 static void register_scripted_rule(rule_t const &rule)
1557 {
1558  ref_ptr<rule_t> r(rule);
1559  for (string_list::const_iterator i = rule.targets.begin(),
1560  i_end = rule.targets.end(); i != i_end; ++i)
1561  {
1562  std::pair<rule_map::iterator, bool> j =
1563  specific_rules.insert(std::make_pair(*i, r));
1564  if (j.second) continue;
1565  std::cerr << "Failed to load rules: " << *i
1566  << " cannot be the target of several rules" << std::endl;
1567  exit(EXIT_FAILURE);
1568  }
1569 
1571  dep->targets = rule.targets;
1572  dep->deps.insert(rule.deps.begin(), rule.deps.end());
1573  for (string_list::const_iterator i = rule.targets.begin(),
1574  i_end = rule.targets.end(); i != i_end; ++i)
1575  {
1577  dep->deps.insert(d->deps.begin(), d->deps.end());
1578  d = dep;
1579  }
1580 }
1581 
1582 /**
1583  * Read a rule starting with target @a first, if nonempty.
1584  * Store into #generic_rules or #specific_rules depending on its genericity.
1585  */
1586 static void load_rule(std::istream &in, std::string const &first)
1587 {
1588  DEBUG_open << "Reading rule for target " << first << "... ";
1589  if (false)
1590  {
1591  error:
1592  DEBUG_close << "failed\n";
1593  std::cerr << "Failed to load rules: syntax error" << std::endl;
1594  exit(EXIT_FAILURE);
1595  }
1596  rule_t rule;
1597 
1598  // Read targets and check genericity.
1599  string_list targets;
1600  if (!read_words(in, targets)) goto error;
1601  if (!first.empty()) targets.push_front(first);
1602  else if (targets.empty()) goto error;
1603  else DEBUG << "actual target: " << targets.front() << std::endl;
1604  bool generic = false;
1605  normalize_list(targets, "", "");
1606  for (string_list::const_iterator i = targets.begin(),
1607  i_end = targets.end(); i != i_end; ++i)
1608  {
1609  if (i->empty()) goto error;
1610  if ((i->find('%') != std::string::npos) != generic)
1611  {
1612  if (i == targets.begin()) generic = true;
1613  else goto error;
1614  }
1615  }
1616  std::swap(rule.targets, targets);
1617  skip_spaces(in);
1618  if (in.get() != ':') goto error;
1619 
1620  bool assignment = false;
1621 
1622  // Read dependencies.
1623  {
1624  string_list v;
1625  if (expect_token(in, Word))
1626  {
1627  std::string d = read_word(in);
1628  if (int tok = expect_token(in, Equal | Plusequal))
1629  {
1630  if (!read_words(in, v)) goto error;
1631  assign_t &a = rule.assigns[d];
1632  a.append = tok == Plusequal;
1633  a.value.swap(v);
1634  assignment = true;
1635  goto end_line;
1636  }
1637  v.push_back(d);
1638  }
1639 
1640  if (!read_words(in, v)) goto error;
1641  normalize_list(v, "", "");
1642  rule.deps.swap(v);
1643 
1644  if (expect_token(in, Pipe))
1645  {
1646  if (!read_words(in, v)) goto error;
1647  normalize_list(v, "", "");
1648  rule.wdeps.swap(v);
1649  }
1650  }
1651 
1652  end_line:
1653  skip_spaces(in);
1654  if (!skip_eol(in, true)) goto error;
1655 
1656  // Read script.
1657  std::ostringstream buf;
1658  while (true)
1659  {
1660  char c = in.get();
1661  if (!in.good()) break;
1662  if (c == '\t' || c == ' ')
1663  {
1664  in.get(*buf.rdbuf());
1665  if (in.fail() && !in.eof()) in.clear();
1666  }
1667  else if (c == '\r' || c == '\n')
1668  buf << c;
1669  else
1670  {
1671  in.putback(c);
1672  break;
1673  }
1674  }
1675  rule.script = buf.str();
1676 
1677  // Register phony targets.
1678  if (rule.targets.front() == ".PHONY")
1679  {
1680  for (string_list::const_iterator i = rule.deps.begin(),
1681  i_end = rule.deps.end(); i != i_end; ++i)
1682  {
1683  status[*i].status = Todo;
1684  }
1685  return;
1686  }
1687 
1688  // Add generic rules to the correct set.
1689  if (generic)
1690  {
1691  if (assignment) goto error;
1692  generic_rules.push_back(rule);
1693  return;
1694  }
1695 
1696  if (!rule.script.empty())
1697  {
1698  if (assignment) goto error;
1699  register_scripted_rule(rule);
1700  }
1701  else
1702  {
1703  // Swap away the targets to avoid costly copies when registering.
1704  string_list targets;
1705  std::swap(rule.targets, targets);
1706  register_transparent_rule(rule, targets);
1707  std::swap(rule.targets, targets);
1708  }
1709 
1710  // If there is no default target yet, mark it as such.
1711  if (first_target.empty())
1712  first_target = rule.targets.front();
1713 }
1714 
1715 /**
1716  * Load rules from @a remakefile.
1717  * If some rules have dependencies and non-generic targets, add these
1718  * dependencies to the targets.
1719  */
1720 static void load_rules(std::string const &remakefile)
1721 {
1722  DEBUG_open << "Loading rules... ";
1723  if (false)
1724  {
1725  error:
1726  std::cerr << "Failed to load rules: syntax error" << std::endl;
1727  exit(EXIT_FAILURE);
1728  }
1729  std::ifstream in(remakefile.c_str());
1730  if (!in.good())
1731  {
1732  std::cerr << "Failed to load rules: no Remakefile found" << std::endl;
1733  exit(EXIT_FAILURE);
1734  }
1735  skip_empty(in);
1736 
1737  string_list options;
1738 
1739  // Read rules
1740  while (in.good())
1741  {
1742  char c = in.peek();
1743  if (c == '#')
1744  {
1745  while (in.get() != '\n') {}
1746  skip_empty(in);
1747  continue;
1748  }
1749  if (c == ' ' || c == '\t') goto error;
1750  if (expect_token(in, Word))
1751  {
1752  std::string name = read_word(in);
1753  if (name.empty()) goto error;
1754  if (int tok = expect_token(in, Equal | Plusequal))
1755  {
1756  DEBUG << "Assignment to variable " << name << std::endl;
1757  string_list value;
1758  if (!read_words(in, value)) goto error;
1759  string_list &dest =
1760  *(name == ".OPTIONS" ? &options : &variables[name]);
1761  if (tok == Equal) dest.swap(value);
1762  else dest.splice(dest.end(), value);
1763  if (!skip_eol(in, true)) goto error;
1764  }
1765  else load_rule(in, name);
1766  }
1767  else load_rule(in, std::string());
1768  }
1769 
1770  // Set actual options.
1771  for (string_list::const_iterator i = options.begin(),
1772  i_end = options.end(); i != i_end; ++i)
1773  {
1774  if (*i == "variable-propagation") propagate_vars = true;
1775  else
1776  {
1777  std::cerr << "Failed to load rules: unrecognized option" << std::endl;
1778  exit(EXIT_FAILURE);
1779  }
1780  }
1781 }
1782 
1783 /** @} */
1784 
1785 /**
1786  * @defgroup rules Rule resolution
1787  *
1788  * @{
1789  */
1790 
1791 static void merge_rule(rule_t &dest, rule_t const &src)
1792 {
1793  dest.deps.insert(dest.deps.end(), src.deps.begin(), src.deps.end());
1794  dest.wdeps.insert(dest.wdeps.end(), src.wdeps.begin(), src.wdeps.end());
1795  for (assign_map::const_iterator i = src.assigns.begin(),
1796  i_end = src.assigns.end(); i != i_end; ++i)
1797  {
1798  if (!i->second.append)
1799  {
1800  new_assign:
1801  dest.assigns[i->first] = i->second;
1802  continue;
1803  }
1804  assign_map::iterator j = dest.assigns.find(i->first);
1805  if (j == dest.assigns.end()) goto new_assign;
1806  j->second.value.insert(j->second.value.end(),
1807  i->second.value.begin(), i->second.value.end());
1808  }
1809 }
1810 
1811 /**
1812  * Substitute a pattern into a list of strings.
1813  */
1814 static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
1815 {
1816  for (string_list::const_iterator i = src.begin(),
1817  i_end = src.end(); i != i_end; ++i)
1818  {
1819  size_t pos = i->find('%');
1820  if (pos == std::string::npos) dst.push_back(*i);
1821  else dst.push_back(i->substr(0, pos) + pat + i->substr(pos + 1));
1822  }
1823 }
1824 
1825 /**
1826  * Find a generic rule matching @a target:
1827  * - the one leading to shorter matches has priority,
1828  * - among equivalent rules, the earliest one has priority.
1829  */
1830 static void find_generic_rule(job_t &job, std::string const &target)
1831 {
1832  size_t tlen = target.length(), plen = tlen + 1;
1833  for (rule_list::const_iterator i = generic_rules.begin(),
1834  i_end = generic_rules.end(); i != i_end; ++i)
1835  {
1836  for (string_list::const_iterator j = i->targets.begin(),
1837  j_end = i->targets.end(); j != j_end; ++j)
1838  {
1839  size_t len = j->length();
1840  if (tlen < len) continue;
1841  if (plen <= tlen - (len - 1)) continue;
1842  size_t pos = j->find('%');
1843  if (pos == std::string::npos) continue;
1844  size_t len2 = len - (pos + 1);
1845  if (j->compare(0, pos, target, 0, pos) ||
1846  j->compare(pos + 1, len2, target, tlen - len2, len2))
1847  continue;
1848  plen = tlen - (len - 1);
1849  job.stem = target.substr(pos, plen);
1850  job.rule = rule_t();
1851  job.rule.script = i->script;
1852  substitute_pattern(job.stem, i->targets, job.rule.targets);
1853  substitute_pattern(job.stem, i->deps, job.rule.deps);
1854  substitute_pattern(job.stem, i->wdeps, job.rule.wdeps);
1855  break;
1856  }
1857  }
1858 }
1859 
1860 /**
1861  * Find a specific rule matching @a target. Return a generic one otherwise.
1862  * If there is both a specific rule with an empty script and a generic rule, the
1863  * generic one is returned after adding the dependencies of the specific one.
1864  */
1865 static void find_rule(job_t &job, std::string const &target)
1866 {
1867  rule_map::const_iterator i = specific_rules.find(target),
1868  i_end = specific_rules.end();
1869  // If there is a specific rule with a script, return it.
1870  if (i != i_end && !i->second->script.empty())
1871  {
1872  job.rule = *i->second;
1873  return;
1874  }
1875  find_generic_rule(job, target);
1876  // If there is no generic rule, return the specific rule (no script), if any.
1877  if (job.rule.targets.empty())
1878  {
1879  if (i != i_end)
1880  {
1881  job.rule = *i->second;
1882  return;
1883  }
1884  }
1885  // Optimize the lookup when there is only one target (already looked up).
1886  if (job.rule.targets.size() == 1)
1887  {
1888  if (i == i_end) return;
1889  merge_rule(job.rule, *i->second);
1890  return;
1891  }
1892  // Add the dependencies of the specific rules of every target to the
1893  // generic rule. If any of those rules has a nonempty script, error out.
1894  for (string_list::const_iterator j = job.rule.targets.begin(),
1895  j_end = job.rule.targets.end(); j != j_end; ++j)
1896  {
1897  i = specific_rules.find(*j);
1898  if (i == i_end) continue;
1899  if (!i->second->script.empty()) return;
1900  merge_rule(job.rule, *i->second);
1901  }
1902 }
1903 
1904 /** @} */
1905 
1906 /**
1907  * @defgroup status Target status
1908  *
1909  * @{
1910  */
1911 
1912 /**
1913  * Compute and memoize the status of @a target:
1914  * - if the file does not exist, the target is obsolete,
1915  * - if any dependency is obsolete or younger than the file, it is obsolete,
1916  * - otherwise it is up-to-date.
1917  *
1918  * @note For rules with multiple targets, all the targets share the same
1919  * status. (If one is obsolete, they all are.) The second rule above
1920  * is modified in that case: the latest target is chosen, not the oldest!
1921  */
1922 static status_t const &get_status(std::string const &target)
1923 {
1924  std::pair<status_map::iterator,bool> i =
1925  status.insert(std::make_pair(target, status_t()));
1926  status_t &ts = i.first->second;
1927  if (!i.second) return ts;
1928  DEBUG_open << "Checking status of " << target << "... ";
1929  dependency_map::const_iterator j = dependencies.find(target);
1930  if (j == dependencies.end())
1931  {
1932  struct stat s;
1933  if (stat(target.c_str(), &s) != 0)
1934  {
1935  DEBUG_close << "missing\n";
1936  ts.status = Todo;
1937  ts.last = 0;
1938  return ts;
1939  }
1940  DEBUG_close << "up-to-date\n";
1941  ts.status = Uptodate;
1942  ts.last = s.st_mtime;
1943  return ts;
1944  }
1945  if (obsolete_targets)
1946  {
1947  DEBUG_close << "forcefully obsolete\n";
1948  ts.status = Todo;
1949  ts.last = 0;
1950  return ts;
1951  }
1952  dependency_t const &dep = *j->second;
1953  status_e st = Uptodate;
1954  time_t latest = 0;
1955  for (string_list::const_iterator k = dep.targets.begin(),
1956  k_end = dep.targets.end(); k != k_end; ++k)
1957  {
1958  struct stat s;
1959  if (stat(k->c_str(), &s) != 0)
1960  {
1961  if (st == Uptodate) DEBUG_close << *k << " missing\n";
1962  s.st_mtime = 0;
1963  st = Todo;
1964  }
1965  status[*k].last = s.st_mtime;
1966  if (s.st_mtime > latest) latest = s.st_mtime;
1967  }
1968  if (st != Uptodate) goto update;
1969  for (string_set::const_iterator k = dep.deps.begin(),
1970  k_end = dep.deps.end(); k != k_end; ++k)
1971  {
1972  status_t const &ts_ = get_status(*k);
1973  if (latest < ts_.last)
1974  {
1975  DEBUG_close << "older than " << *k << std::endl;
1976  st = Todo;
1977  goto update;
1978  }
1979  if (ts_.status != Uptodate && st != Recheck)
1980  {
1981  DEBUG << "obsolete dependency " << *k << std::endl;
1982  st = Recheck;
1983  }
1984  }
1985  if (st == Uptodate) DEBUG_close << "all siblings up-to-date\n";
1986  update:
1987  for (string_list::const_iterator k = dep.targets.begin(),
1988  k_end = dep.targets.end(); k != k_end; ++k)
1989  {
1990  status[*k].status = st;
1991  }
1992  return ts;
1993 }
1994 
1995 /**
1996  * Change the status of @a target to #Remade or #Uptodate depending on whether
1997  * its modification time changed.
1998  */
1999 static void update_status(std::string const &target)
2000 {
2001  DEBUG_open << "Rechecking status of " << target << "... ";
2002  status_map::iterator i = status.find(target);
2003  assert(i != status.end());
2004  status_t &ts = i->second;
2005  ts.status = Remade;
2006  if (ts.last >= now)
2007  {
2008  DEBUG_close << "possibly remade\n";
2009  return;
2010  }
2011  struct stat s;
2012  if (stat(target.c_str(), &s) != 0)
2013  {
2014  DEBUG_close << "missing\n";
2015  ts.last = 0;
2016  }
2017  else if (s.st_mtime != ts.last)
2018  {
2019  DEBUG_close << "remade\n";
2020  ts.last = s.st_mtime;
2021  }
2022  else
2023  {
2024  DEBUG_close << "unchanged\n";
2025  ts.status = Uptodate;
2026  }
2027 }
2028 
2029 /**
2030  * Check whether all the prerequisites of @a target ended being up-to-date.
2031  */
2032 static bool still_need_rebuild(std::string const &target)
2033 {
2034  status_map::const_iterator i = status.find(target);
2035  assert(i != status.end());
2036  if (i->second.status != RunningRecheck) return true;
2037  DEBUG_open << "Rechecking obsoleteness of " << target << "... ";
2038  dependency_map::const_iterator j = dependencies.find(target);
2039  assert(j != dependencies.end());
2040  dependency_t const &dep = *j->second;
2041  for (string_set::const_iterator k = dep.deps.begin(),
2042  k_end = dep.deps.end(); k != k_end; ++k)
2043  {
2044  if (status[*k].status != Uptodate) return true;
2045  }
2046  for (string_list::const_iterator k = dep.targets.begin(),
2047  k_end = dep.targets.end(); k != k_end; ++k)
2048  {
2049  status[*k].status = Uptodate;
2050  }
2051  DEBUG_close << "no longer obsolete\n";
2052  return false;
2053 }
2054 
2055 /** @} */
2056 
2057 /**
2058  * @defgroup server Server
2059  *
2060  * @{
2061  */
2062 
2063 /**
2064  * Handle job completion.
2065  */
2066 static void complete_job(int job_id, bool success, bool started = true)
2067 {
2068  DEBUG << "Completing job " << job_id << '\n';
2069  job_map::iterator i = jobs.find(job_id);
2070  assert(i != jobs.end());
2071  string_list const &targets = i->second.rule.targets;
2072  if (success)
2073  {
2074  bool show = show_targets && started;
2075  if (show) std::cout << "Finished";
2076  for (string_list::const_iterator j = targets.begin(),
2077  j_end = targets.end(); j != j_end; ++j)
2078  {
2079  update_status(*j);
2080  if (show) std::cout << ' ' << *j;
2081  }
2082  if (show) std::cout << std::endl;
2083  }
2084  else
2085  {
2086  std::cerr << "Failed to build";
2087  for (string_list::const_iterator j = targets.begin(),
2088  j_end = targets.end(); j != j_end; ++j)
2089  {
2090  std::cerr << ' ' << *j;
2091  update_status(*j);
2092  status_e &s = status[*j].status;
2093  if (s != Uptodate)
2094  {
2095  DEBUG << "Removing " << *j << '\n';
2096  remove(j->c_str());
2097  }
2098  s = Failed;
2099  }
2100  std::cerr << std::endl;
2101  }
2102  jobs.erase(i);
2103 }
2104 
2105 /**
2106  * Return the script obtained by substituting variables.
2107  */
2108 static std::string prepare_script(job_t const &job)
2109 {
2110  std::string const &s = job.rule.script;
2111  std::istringstream in(s);
2112  std::ostringstream out;
2113  size_t len = s.size();
2114 
2115  while (!in.eof())
2116  {
2117  size_t pos = in.tellg(), p = s.find('$', pos);
2118  if (p == std::string::npos || p == len - 1) p = len;
2119  out.write(&s[pos], p - pos);
2120  if (p == len) break;
2121  ++p;
2122  switch (s[p])
2123  {
2124  case '$':
2125  out << '$';
2126  in.seekg(p + 1);
2127  break;
2128  case '<':
2129  if (!job.rule.deps.empty())
2130  out << job.rule.deps.front();
2131  in.seekg(p + 1);
2132  break;
2133  case '^':
2134  {
2135  bool first = true;
2136  for (string_list::const_iterator i = job.rule.deps.begin(),
2137  i_end = job.rule.deps.end(); i != i_end; ++i)
2138  {
2139  if (first) first = false;
2140  else out << ' ';
2141  out << *i;
2142  }
2143  in.seekg(p + 1);
2144  break;
2145  }
2146  case '@':
2147  assert(!job.rule.targets.empty());
2148  out << job.rule.targets.front();
2149  in.seekg(p + 1);
2150  break;
2151  case '*':
2152  out << job.stem;
2153  in.seekg(p + 1);
2154  break;
2155  case '(':
2156  {
2157  in.seekg(p - 1);
2158  bool first = true;
2159  input_generator gen(in, &job.vars, true);
2160  while (true)
2161  {
2162  std::string w;
2163  input_status s = gen.next(w);
2164  if (s == SyntaxError)
2165  {
2166  // TODO
2167  return "false";
2168  }
2169  if (s == Eof) break;
2170  if (first) first = false;
2171  else out << ' ';
2172  out << w;
2173  }
2174  break;
2175  }
2176  default:
2177  // Let dollars followed by an unrecognized character
2178  // go through. This differs from Make, which would
2179  // use a one-letter variable.
2180  out << '$';
2181  in.seekg(p);
2182  }
2183  }
2184 
2185  return out.str();
2186 }
2187 
2188 /**
2189  * Execute the script from @a rule.
2190  */
2191 static status_e run_script(int job_id, job_t const &job)
2192 {
2194  dep->targets = job.rule.targets;
2195  dep->deps.insert(job.rule.deps.begin(), job.rule.deps.end());
2196  if (show_targets) std::cout << "Building";
2197  for (string_list::const_iterator i = job.rule.targets.begin(),
2198  i_end = job.rule.targets.end(); i != i_end; ++i)
2199  {
2200  dependencies[*i] = dep;
2201  if (show_targets) std::cout << ' ' << *i;
2202  }
2203  if (show_targets) std::cout << std::endl;
2204 
2205  std::string script = prepare_script(job);
2206 
2207  std::ostringstream job_id_buf;
2208  job_id_buf << job_id;
2209  std::string job_id_ = job_id_buf.str();
2210 
2211  DEBUG_open << "Starting script for job " << job_id << "... ";
2212  if (script.empty())
2213  {
2214  DEBUG_close << "no script\n";
2215  complete_job(job_id, true);
2216  return Remade;
2217  }
2218 
2219  if (false)
2220  {
2221  error:
2222  DEBUG_close << "failed\n";
2223  complete_job(job_id, false);
2224  return Failed;
2225  }
2226 
2227 #ifdef WINDOWS
2228  HANDLE pfd[2];
2229  if (false)
2230  {
2231  error2:
2232  CloseHandle(pfd[0]);
2233  CloseHandle(pfd[1]);
2234  goto error;
2235  }
2236  if (!CreatePipe(&pfd[0], &pfd[1], NULL, 0))
2237  goto error;
2238  if (!SetHandleInformation(pfd[0], HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT))
2239  goto error2;
2240  STARTUPINFO si;
2241  ZeroMemory(&si, sizeof(STARTUPINFO));
2242  si.cb = sizeof(STARTUPINFO);
2243  si.hStdError = GetStdHandle(STD_ERROR_HANDLE);
2244  si.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
2245  si.hStdInput = pfd[0];
2246  si.dwFlags |= STARTF_USESTDHANDLES;
2247  PROCESS_INFORMATION pi;
2248  ZeroMemory(&pi, sizeof(PROCESS_INFORMATION));
2249  if (!SetEnvironmentVariable("REMAKE_JOB_ID", job_id_.c_str()))
2250  goto error2;
2251  char const *argv = echo_scripts ? "SH.EXE -e -s -v" : "SH.EXE -e -s";
2252  if (!CreateProcess(NULL, (char *)argv, NULL, NULL,
2253  true, 0, NULL, NULL, &si, &pi))
2254  {
2255  goto error2;
2256  }
2257  CloseHandle(pi.hThread);
2258  DWORD len = script.length(), wlen;
2259  if (!WriteFile(pfd[1], script.c_str(), len, &wlen, NULL) || wlen < len)
2260  std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2261  CloseHandle(pfd[0]);
2262  CloseHandle(pfd[1]);
2263  ++running_jobs;
2264  job_pids[pi.hProcess] = job_id;
2265  return Running;
2266 #else
2267  int pfd[2];
2268  if (false)
2269  {
2270  error2:
2271  close(pfd[0]);
2272  close(pfd[1]);
2273  goto error;
2274  }
2275  if (pipe(pfd) == -1)
2276  goto error;
2277  if (setenv("REMAKE_JOB_ID", job_id_.c_str(), 1))
2278  goto error2;
2279  if (pid_t pid = vfork())
2280  {
2281  if (pid == -1) goto error2;
2282  ssize_t len = script.length();
2283  if (write(pfd[1], script.c_str(), len) < len)
2284  std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2285  close(pfd[0]);
2286  close(pfd[1]);
2287  ++running_jobs;
2288  job_pids[pid] = job_id;
2289  return Running;
2290  }
2291  // Child process starts here. Notice the use of vfork above.
2292  char const *argv[5] = { "sh", "-e", "-s", NULL, NULL };
2293  if (echo_scripts) argv[3] = "-v";
2294  close(pfd[1]);
2295  if (pfd[0] != 0)
2296  {
2297  dup2(pfd[0], 0);
2298  close(pfd[0]);
2299  }
2300  execve("/bin/sh", (char **)argv, environ);
2301  _exit(EXIT_FAILURE);
2302 #endif
2303 }
2304 
2305 /**
2306  * Create a job for @a target according to the loaded rules.
2307  * Mark all the targets from the rule as running and reset their dependencies.
2308  * Inherit variables from @a current, if enabled.
2309  * If the rule has dependencies, create a new client to build them just
2310  * before @a current, and change @a current so that it points to it.
2311  */
2312 static status_e start(std::string const &target, client_list::iterator &current)
2313 {
2314  int job_id = job_counter++;
2315  DEBUG_open << "Starting job " << job_id << " for " << target << "... ";
2316  job_t &job = jobs[job_id];
2317  find_rule(job, target);
2318  if (job.rule.targets.empty())
2319  {
2320  status[target].status = Failed;
2321  DEBUG_close << "failed\n";
2322  std::cerr << "No rule for building " << target << std::endl;
2323  return Failed;
2324  }
2325  bool has_deps = !job.rule.deps.empty() || !job.rule.wdeps.empty();
2326  status_e st = Running;
2327  if (has_deps && status[target].status == Recheck)
2328  st = RunningRecheck;
2329  for (string_list::const_iterator i = job.rule.targets.begin(),
2330  i_end = job.rule.targets.end(); i != i_end; ++i)
2331  {
2332  status[*i].status = st;
2333  }
2334  if (propagate_vars) job.vars = current->vars;
2335  for (assign_map::const_iterator i = job.rule.assigns.begin(),
2336  i_end = job.rule.assigns.end(); i != i_end; ++i)
2337  {
2338  std::pair<variable_map::iterator, bool> k =
2339  job.vars.insert(std::make_pair(i->first, string_list()));
2340  string_list &v = k.first->second;
2341  if (i->second.append)
2342  {
2343  if (k.second)
2344  {
2345  variable_map::const_iterator j = variables.find(i->first);
2346  if (j != variables.end()) v = j->second;
2347  }
2348  }
2349  else if (!k.second) v.clear();
2350  v.insert(v.end(), i->second.value.begin(), i->second.value.end());
2351  }
2352  if (has_deps)
2353  {
2354  current = clients.insert(current, client_t());
2355  current->job_id = job_id;
2356  current->pending = job.rule.deps;
2357  current->pending.insert(current->pending.end(),
2358  job.rule.wdeps.begin(), job.rule.wdeps.end());
2359  if (propagate_vars) current->vars = job.vars;
2360  current->delayed = true;
2361  return RunningRecheck;
2362  }
2363  return run_script(job_id, job);
2364 }
2365 
2366 /**
2367  * Send a reply to a client then remove it.
2368  * If the client was a dependency client, start the actual script.
2369  */
2370 static void complete_request(client_t &client, bool success)
2371 {
2372  DEBUG_open << "Completing request from client of job " << client.job_id << "... ";
2373  if (client.delayed)
2374  {
2375  assert(client.socket == INVALID_SOCKET);
2376  if (success)
2377  {
2378  job_map::const_iterator i = jobs.find(client.job_id);
2379  assert(i != jobs.end());
2380  if (still_need_rebuild(i->second.rule.targets.front()))
2381  run_script(client.job_id, i->second);
2382  else complete_job(client.job_id, true, false);
2383  }
2384  else complete_job(client.job_id, false);
2385  }
2386  else if (client.socket != INVALID_SOCKET)
2387  {
2388  char res = success ? 1 : 0;
2389  send(client.socket, &res, 1, MSG_NOSIGNAL);
2390  #ifdef WINDOWS
2391  closesocket(client.socket);
2392  #else
2393  close(client.socket);
2394  #endif
2395  --waiting_jobs;
2396  }
2397 
2398  if (client.job_id < 0 && !success) build_failure = true;
2399 }
2400 
2401 /**
2402  * Return whether there are slots for starting new jobs.
2403  */
2404 static bool has_free_slots()
2405 {
2406  if (max_active_jobs <= 0) return true;
2408 }
2409 
2410 /**
2411  * Handle client requests:
2412  * - check for running targets that have finished,
2413  * - start as many pending targets as allowed,
2414  * - complete the request if there are neither running nor pending targets
2415  * left or if any of them failed.
2416  *
2417  * @return true if some child processes are still running.
2418  *
2419  * @post If there are pending requests, at least one child process is running.
2420  *
2421  * @invariant New free slots cannot appear during a run, since the only way to
2422  * decrease #running_jobs is #finalize_job and the only way to
2423  * increase #waiting_jobs is #accept_client. None of these functions
2424  * are called during a run. So breaking out as soon as there no free
2425  * slots left is fine.
2426  */
2427 static bool handle_clients()
2428 {
2429  DEBUG_open << "Handling client requests... ";
2430  restart:
2431  bool need_restart = false;
2432 
2433  for (client_list::iterator i = clients.begin(), i_next = i,
2434  i_end = clients.end(); i != i_end && has_free_slots(); i = i_next)
2435  {
2436  ++i_next;
2437  DEBUG_open << "Handling client from job " << i->job_id << "... ";
2438 
2439  // Remove running targets that have finished.
2440  for (string_set::iterator j = i->running.begin(), j_next = j,
2441  j_end = i->running.end(); j != j_end; j = j_next)
2442  {
2443  ++j_next;
2444  status_map::const_iterator k = status.find(*j);
2445  assert(k != status.end());
2446  switch (k->second.status)
2447  {
2448  case Running:
2449  case RunningRecheck:
2450  break;
2451  case Failed:
2452  i->failed = true;
2453  if (!keep_going) goto complete;
2454  // no break
2455  case Uptodate:
2456  case Remade:
2457  i->running.erase(j);
2458  break;
2459  case Recheck:
2460  case Todo:
2461  assert(false);
2462  }
2463  }
2464 
2465  // Start pending targets.
2466  while (!i->pending.empty())
2467  {
2468  std::string target = i->pending.front();
2469  i->pending.pop_front();
2470  switch (get_status(target).status)
2471  {
2472  case Running:
2473  case RunningRecheck:
2474  i->running.insert(target);
2475  break;
2476  case Failed:
2477  pending_failed:
2478  i->failed = true;
2479  if (!keep_going) goto complete;
2480  // no break
2481  case Uptodate:
2482  case Remade:
2483  break;
2484  case Recheck:
2485  case Todo:
2486  client_list::iterator j = i;
2487  switch (start(target, i))
2488  {
2489  case Failed:
2490  goto pending_failed;
2491  case Running:
2492  // A shell was started, check for free slots.
2493  j->running.insert(target);
2494  if (!has_free_slots()) return true;
2495  break;
2496  case RunningRecheck:
2497  // Switch to the dependency client that was inserted.
2498  j->running.insert(target);
2499  i_next = j;
2500  break;
2501  case Remade:
2502  // Nothing to run.
2503  need_restart = true;
2504  break;
2505  default:
2506  assert(false);
2507  }
2508  }
2509  }
2510 
2511  // Try to complete the request.
2512  // (This might start a new job if it was a dependency client.)
2513  if (i->running.empty() || i->failed)
2514  {
2515  complete:
2516  complete_request(*i, !i->failed);
2517  DEBUG_close << (i->failed ? "failed\n" : "finished\n");
2518  clients.erase(i);
2519  need_restart = true;
2520  }
2521  }
2522 
2523  if (running_jobs != waiting_jobs) return true;
2524  if (running_jobs == 0 && clients.empty()) return false;
2525  if (need_restart) goto restart;
2526 
2527  // There is a circular dependency.
2528  // Try to break it by completing one of the requests.
2529  assert(!clients.empty());
2530  std::cerr << "Circular dependency detected" << std::endl;
2531  client_list::iterator i = clients.begin();
2532  complete_request(*i, false);
2533  clients.erase(i);
2534  goto restart;
2535 }
2536 
2537 /**
2538  * Create a named unix socket that listens for build requests. Also set
2539  * the REMAKE_SOCKET environment variable that will be inherited by all
2540  * the job scripts.
2541  */
2542 static void create_server()
2543 {
2544  if (false)
2545  {
2546  error:
2547  perror("Failed to create server");
2548 #ifndef WINDOWS
2549  error2:
2550 #endif
2551  exit(EXIT_FAILURE);
2552  }
2553  DEBUG_open << "Creating server... ";
2554 
2555 #ifdef WINDOWS
2556  // Prepare a windows socket.
2557  struct sockaddr_in socket_addr;
2558  socket_addr.sin_family = AF_INET;
2559  socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2560  socket_addr.sin_port = 0;
2561 
2562  // Create and listen to the socket.
2563  socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2564  if (socket_fd == INVALID_SOCKET) goto error;
2565  if (!SetHandleInformation((HANDLE)socket_fd, HANDLE_FLAG_INHERIT, 0))
2566  goto error;
2567  if (bind(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2568  goto error;
2569  int len = sizeof(sockaddr_in);
2570  if (getsockname(socket_fd, (struct sockaddr *)&socket_addr, &len))
2571  goto error;
2572  std::ostringstream buf;
2573  buf << socket_addr.sin_port;
2574  if (!SetEnvironmentVariable("REMAKE_SOCKET", buf.str().c_str()))
2575  goto error;
2576  if (listen(socket_fd, 1000)) goto error;
2577 #else
2578  // Set signal handlers for SIGCHLD and SIGINT.
2579  // Block SIGCHLD (unblocked during select).
2580  sigset_t sigmask;
2581  sigemptyset(&sigmask);
2582  sigaddset(&sigmask, SIGCHLD);
2583  if (sigprocmask(SIG_BLOCK, &sigmask, NULL) == -1) goto error;
2584  struct sigaction sa;
2585  sa.sa_flags = 0;
2586  sigemptyset(&sa.sa_mask);
2587  sa.sa_handler = &sigchld_handler;
2588  if (sigaction(SIGCHLD, &sa, NULL) == -1) goto error;
2589  sa.sa_handler = &sigint_handler;
2590  if (sigaction(SIGINT, &sa, NULL) == -1) goto error;
2591 
2592  // Prepare a named unix socket in temporary directory.
2593  socket_name = tempnam(NULL, "rmk-");
2594  if (!socket_name) goto error2;
2595  struct sockaddr_un socket_addr;
2596  size_t len = strlen(socket_name);
2597  if (len >= sizeof(socket_addr.sun_path) - 1) goto error2;
2598  socket_addr.sun_family = AF_UNIX;
2599  strcpy(socket_addr.sun_path, socket_name);
2600  len += sizeof(socket_addr.sun_family);
2601  if (setenv("REMAKE_SOCKET", socket_name, 1)) goto error;
2602 
2603  // Create and listen to the socket.
2604 #ifdef LINUX
2605  socket_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
2606  if (socket_fd == INVALID_SOCKET) goto error;
2607 #else
2608  socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2609  if (socket_fd == INVALID_SOCKET) goto error;
2610  if (fcntl(socket_fd, F_SETFD, FD_CLOEXEC) < 0) goto error;
2611 #endif
2612  if (bind(socket_fd, (struct sockaddr *)&socket_addr, len))
2613  goto error;
2614  if (listen(socket_fd, 1000)) goto error;
2615 #endif
2616 }
2617 
2618 /**
2619  * Accept a connection from a client, get the job it spawned from,
2620  * get the targets, and mark them as dependencies of the job targets.
2621  */
2622 static void accept_client()
2623 {
2624  DEBUG_open << "Handling client request... ";
2625 
2626  // Accept connection.
2627 #ifdef WINDOWS
2628  socket_t fd = accept(socket_fd, NULL, NULL);
2629  if (fd == INVALID_SOCKET) return;
2630  if (!SetHandleInformation((HANDLE)fd, HANDLE_FLAG_INHERIT, 0))
2631  {
2632  error2:
2633  std::cerr << "Unexpected failure while setting connection with client" << std::endl;
2634  closesocket(fd);
2635  return;
2636  }
2637  // WSAEventSelect puts sockets into nonblocking mode, so disable it here.
2638  u_long nbio = 0;
2639  if (ioctlsocket(fd, FIONBIO, &nbio)) goto error2;
2640 #elif defined(LINUX)
2641  int fd = accept4(socket_fd, NULL, NULL, SOCK_CLOEXEC);
2642  if (fd < 0) return;
2643 #else
2644  int fd = accept(socket_fd, NULL, NULL);
2645  if (fd < 0) return;
2646  if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0) return;
2647 #endif
2648  clients.push_front(client_t());
2649  client_list::iterator proc = clients.begin();
2650 
2651  if (false)
2652  {
2653  error:
2654  DEBUG_close << "failed\n";
2655  std::cerr << "Received an ill-formed client message" << std::endl;
2656  #ifdef WINDOWS
2657  closesocket(fd);
2658  #else
2659  close(fd);
2660  #endif
2661  clients.erase(proc);
2662  return;
2663  }
2664 
2665  // Receive message. Stop when encountering two nuls in a row.
2666  std::vector<char> buf;
2667  size_t len = 0;
2668  while (len < sizeof(int) + 2 || buf[len - 1] || buf[len - 2])
2669  {
2670  buf.resize(len + 1024);
2671  ssize_t l = recv(fd, &buf[0] + len, 1024, 0);
2672  if (l <= 0) goto error;
2673  len += l;
2674  }
2675 
2676  // Parse job that spawned the client.
2677  int job_id;
2678  memcpy(&job_id, &buf[0], sizeof(int));
2679  proc->socket = fd;
2680  proc->job_id = job_id;
2681  job_map::const_iterator i = jobs.find(job_id);
2682  if (i == jobs.end()) goto error;
2683  DEBUG << "receiving request from job " << job_id << std::endl;
2684  if (propagate_vars) proc->vars = i->second.vars;
2685 
2686  // Parse the targets and the variable assignments.
2687  // Mark the targets as dependencies of the job targets.
2688  dependency_t &dep = *dependencies[i->second.rule.targets.front()];
2689  string_list *last_var = NULL;
2690  char const *p = &buf[0] + sizeof(int);
2691  while (true)
2692  {
2693  len = strlen(p);
2694  if (len == 0)
2695  {
2696  ++waiting_jobs;
2697  break;
2698  }
2699  switch (*p)
2700  {
2701  case 'T':
2702  {
2703  if (len == 1) goto error;
2704  std::string target(p + 1, p + len);
2705  DEBUG << "adding dependency " << target << " to job\n";
2706  proc->pending.push_back(target);
2707  dep.deps.insert(target);
2708  break;
2709  }
2710  case 'V':
2711  {
2712  if (len == 1) goto error;
2713  std::string var(p + 1, p + len);
2714  DEBUG << "adding variable " << var << " to job\n";
2715  last_var = &proc->vars[var];
2716  last_var->clear();
2717  break;
2718  }
2719  case 'W':
2720  {
2721  if (!last_var) goto error;
2722  last_var->push_back(std::string(p + 1, p + len));
2723  break;
2724  }
2725  default:
2726  goto error;
2727  }
2728  p += len + 1;
2729  }
2730 
2731  if (!propagate_vars && !proc->vars.empty())
2732  {
2733  std::cerr << "Assignments are ignored unless 'variable-propagation' is enabled" << std::endl;
2734  proc->vars.clear();
2735  }
2736 }
2737 
2738 /**
2739  * Handle child process exit status.
2740  */
2741 static void finalize_job(pid_t pid, bool res)
2742 {
2743  pid_job_map::iterator i = job_pids.find(pid);
2744  assert(i != job_pids.end());
2745  int job_id = i->second;
2746  job_pids.erase(i);
2747  --running_jobs;
2748  complete_job(job_id, res);
2749 }
2750 
2751 /**
2752  * Loop until all the jobs have finished.
2753  *
2754  * @post There are no client requests left, not even virtual ones.
2755  */
2756 static void server_loop()
2757 {
2758  while (handle_clients())
2759  {
2760  DEBUG_open << "Handling events... ";
2761  #ifdef WINDOWS
2762  size_t len = job_pids.size() + 1;
2763  HANDLE h[len];
2764  int num = 0;
2765  for (pid_job_map::const_iterator i = job_pids.begin(),
2766  i_end = job_pids.end(); i != i_end; ++i, ++num)
2767  {
2768  h[num] = i->first;
2769  }
2770  WSAEVENT aev = WSACreateEvent();
2771  h[num] = aev;
2772  WSAEventSelect(socket_fd, aev, FD_ACCEPT);
2773  DWORD w = WaitForMultipleObjects(len, h, false, INFINITE);
2774  WSAEventSelect(socket_fd, aev, 0);
2775  WSACloseEvent(aev);
2776  if (len <= w)
2777  continue;
2778  if (w == len - 1)
2779  {
2780  accept_client();
2781  continue;
2782  }
2783  pid_t pid = h[w];
2784  DWORD s = 0;
2785  bool res = GetExitCodeProcess(pid, &s) && s == 0;
2786  CloseHandle(pid);
2787  finalize_job(pid, res);
2788  #else
2789  sigset_t emptymask;
2790  sigemptyset(&emptymask);
2791  fd_set fdset;
2792  FD_ZERO(&fdset);
2793  FD_SET(socket_fd, &fdset);
2794  int ret = pselect(socket_fd + 1, &fdset, NULL, NULL, NULL, &emptymask);
2795  if (ret > 0 /* && FD_ISSET(socket_fd, &fdset)*/) accept_client();
2796  if (!got_SIGCHLD) continue;
2797  got_SIGCHLD = 0;
2798  pid_t pid;
2799  int status;
2800  while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
2801  {
2802  bool res = WIFEXITED(status) && WEXITSTATUS(status) == 0;
2803  finalize_job(pid, res);
2804  }
2805  #endif
2806  }
2807 
2808  assert(clients.empty());
2809 }
2810 
2811 /**
2812  * Load dependencies and rules, listen to client requests, and loop until
2813  * all the requests have completed.
2814  * If Remakefile is obsolete, perform a first run with it only, then reload
2815  * the rules, and perform a second with the original clients.
2816  */
2817 static void server_mode(std::string const &remakefile, string_list const &targets)
2818 {
2820  load_rules(remakefile);
2821  create_server();
2822  if (get_status(remakefile).status != Uptodate)
2823  {
2824  clients.push_back(client_t());
2825  clients.back().pending.push_back(remakefile);
2826  server_loop();
2827  if (build_failure) goto early_exit;
2828  variables.clear();
2829  specific_rules.clear();
2830  generic_rules.clear();
2831  first_target.clear();
2832  load_rules(remakefile);
2833  }
2834  clients.push_back(client_t());
2835  if (!targets.empty()) clients.back().pending = targets;
2836  else if (!first_target.empty())
2837  clients.back().pending.push_back(first_target);
2838  server_loop();
2839  early_exit:
2840  close(socket_fd);
2841 #ifndef WINDOWS
2842  remove(socket_name);
2843  free(socket_name);
2844 #endif
2847  {
2848  std::cout << "remake: Leaving directory `" << prefix_dir << '\'' << std::endl;
2849  }
2850  exit(build_failure ? EXIT_FAILURE : EXIT_SUCCESS);
2851 }
2852 
2853 /** @} */
2854 
2855 /**
2856  * @defgroup client Client
2857  *
2858  * @{
2859  */
2860 
2861 /**
2862  * Connect to the server @a socket_name, send a request for building @a targets
2863  * with some @a variables, and exit with the status returned by the server.
2864  */
2865 static void client_mode(char *socket_name, string_list const &targets)
2866 {
2867  if (false)
2868  {
2869  error:
2870  perror("Failed to send targets to server");
2871  exit(EXIT_FAILURE);
2872  }
2873  if (targets.empty()) exit(EXIT_SUCCESS);
2874  DEBUG_open << "Connecting to server... ";
2875 
2876  // Connect to server.
2877 #ifdef WINDOWS
2878  struct sockaddr_in socket_addr;
2879  socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2880  if (socket_fd == INVALID_SOCKET) goto error;
2881  socket_addr.sin_family = AF_INET;
2882  socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2883  socket_addr.sin_port = atoi(socket_name);
2884  if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2885  goto error;
2886 #else
2887  struct sockaddr_un socket_addr;
2888  size_t len = strlen(socket_name);
2889  if (len >= sizeof(socket_addr.sun_path) - 1) exit(EXIT_FAILURE);
2890  socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2891  if (socket_fd == INVALID_SOCKET) goto error;
2892  socket_addr.sun_family = AF_UNIX;
2893  strcpy(socket_addr.sun_path, socket_name);
2894  if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(socket_addr.sun_family) + len))
2895  goto error;
2896 #ifdef MACOSX
2897  int set_option = 1;
2898  if (setsockopt(socket_fd, SOL_SOCKET, SO_NOSIGPIPE, &set_option, sizeof(set_option)))
2899  goto error;
2900 #endif
2901 #endif
2902 
2903  // Send current job id.
2904  char *id = getenv("REMAKE_JOB_ID");
2905  int job_id = id ? atoi(id) : -1;
2906  if (send(socket_fd, (char *)&job_id, sizeof(job_id), MSG_NOSIGNAL) != sizeof(job_id))
2907  goto error;
2908 
2909  // Send targets.
2910  for (string_list::const_iterator i = targets.begin(),
2911  i_end = targets.end(); i != i_end; ++i)
2912  {
2913  DEBUG_open << "Sending target " << *i << "... ";
2914  std::string s = 'T' + *i;
2915  ssize_t len = s.length() + 1;
2916  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2917  goto error;
2918  }
2919 
2920  // Send variables.
2921  for (variable_map::const_iterator i = variables.begin(),
2922  i_end = variables.end(); i != i_end; ++i)
2923  {
2924  DEBUG_open << "Sending variable " << i->first << "... ";
2925  std::string s = 'V' + i->first;
2926  ssize_t len = s.length() + 1;
2927  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2928  goto error;
2929  for (string_list::const_iterator j = i->second.begin(),
2930  j_end = i->second.end(); j != j_end; ++j)
2931  {
2932  std::string s = 'W' + *j;
2933  len = s.length() + 1;
2934  if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2935  goto error;
2936  }
2937  }
2938 
2939  // Send terminating nul and wait for reply.
2940  char result = 0;
2941  if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
2942  if (recv(socket_fd, &result, 1, 0) != 1) exit(EXIT_FAILURE);
2943  exit(result ? EXIT_SUCCESS : EXIT_FAILURE);
2944 }
2945 
2946 /** @} */
2947 
2948 /**
2949  * @defgroup ui User interface
2950  *
2951  * @{
2952  */
2953 
2954 /**
2955  * Display usage and exit with @a exit_status.
2956  */
2957 static void usage(int exit_status)
2958 {
2959  std::cerr << "Usage: remake [options] [target] ...\n"
2960  "Options\n"
2961  " -B, --always-make Unconditionally make all targets.\n"
2962  " -d Echo script commands.\n"
2963  " -d -d Print lots of debugging information.\n"
2964  " -f FILE Read FILE as Remakefile.\n"
2965  " -h, --help Print this message and exit.\n"
2966  " -j[N], --jobs=[N] Allow N jobs at once; infinite jobs with no arg.\n"
2967  " -k, --keep-going Keep going when some targets cannot be made.\n"
2968  " -r Look up targets from the dependencies on stdin.\n"
2969  " -s, --silent, --quiet Do not echo targets.\n";
2970  exit(exit_status);
2971 }
2972 
2973 /**
2974  * This program behaves in two different ways.
2975  *
2976  * - If the environment contains the REMAKE_SOCKET variable, the client
2977  * connects to this socket and sends to the server its build targets.
2978  * It exits once it receives the server reply.
2979  *
2980  * - Otherwise, it creates a server that waits for build requests. It
2981  * also creates a pseudo-client that requests the targets passed on the
2982  * command line.
2983  */
2984 int main(int argc, char *argv[])
2985 {
2986  std::string remakefile;
2987  string_list targets;
2988  bool literal_targets = false;
2989  bool indirect_targets = false;
2990 
2991  // Parse command-line arguments.
2992  for (int i = 1; i < argc; ++i)
2993  {
2994  std::string arg = argv[i];
2995  if (arg.empty()) usage(EXIT_FAILURE);
2996  if (literal_targets) goto new_target;
2997  if (arg == "-h" || arg == "--help") usage(EXIT_SUCCESS);
2998  if (arg == "-d")
2999  if (echo_scripts) debug.active = true;
3000  else echo_scripts = true;
3001  else if (arg == "-k" || arg =="--keep-going")
3002  keep_going = true;
3003  else if (arg == "-s" || arg == "--silent" || arg == "--quiet")
3004  show_targets = false;
3005  else if (arg == "-r")
3006  indirect_targets = true;
3007  else if (arg == "-B" || arg == "--always-make")
3008  obsolete_targets = true;
3009  else if (arg == "-f")
3010  {
3011  if (++i == argc) usage(EXIT_FAILURE);
3012  remakefile = argv[i];
3013  }
3014  else if (arg == "--")
3015  literal_targets = true;
3016  else if (arg.compare(0, 2, "-j") == 0)
3017  max_active_jobs = atoi(arg.c_str() + 2);
3018  else if (arg.compare(0, 7, "--jobs=") == 0)
3019  max_active_jobs = atoi(arg.c_str() + 7);
3020  else
3021  {
3022  if (arg[0] == '-') usage(EXIT_FAILURE);
3023  if (arg.find('=') != std::string::npos)
3024  {
3025  std::istringstream in(arg);
3026  std::string name = read_word(in);
3027  if (name.empty() || !expect_token(in, Equal)) usage(EXIT_FAILURE);
3028  read_words(in, variables[name]);
3029  continue;
3030  }
3031  new_target:
3032  targets.push_back(arg);
3033  DEBUG << "New target: " << arg << '\n';
3034  }
3035  }
3036 
3037  init_working_dir();
3039 
3040  if (indirect_targets)
3041  {
3042  load_dependencies(std::cin);
3043  string_list l;
3044  targets.swap(l);
3045  if (l.empty() && !dependencies.empty())
3046  {
3047  l.push_back(dependencies.begin()->second->targets.front());
3048  }
3049  for (string_list::const_iterator i = l.begin(),
3050  i_end = l.end(); i != i_end; ++i)
3051  {
3052  dependency_map::const_iterator j = dependencies.find(*i);
3053  if (j == dependencies.end()) continue;
3054  dependency_t const &dep = *j->second;
3055  for (string_set::const_iterator k = dep.deps.begin(),
3056  k_end = dep.deps.end(); k != k_end; ++k)
3057  {
3058  targets.push_back(normalize(*k, working_dir, working_dir));
3059  }
3060  }
3061  dependencies.clear();
3062  }
3063 
3064 #ifdef WINDOWS
3065  WSADATA wsaData;
3066  if (WSAStartup(MAKEWORD(2,2), &wsaData))
3067  {
3068  std::cerr << "Unexpected failure while initializing Windows Socket" << std::endl;
3069  return 1;
3070  }
3071 #endif
3072 
3073  // Run as client if REMAKE_SOCKET is present in the environment.
3074  if (char *sn = getenv("REMAKE_SOCKET")) client_mode(sn, targets);
3075 
3076  // Otherwise run as server.
3077  if (remakefile.empty())
3078  {
3079  remakefile = "Remakefile";
3080  init_prefix_dir();
3081  }
3083  server_mode(remakefile, targets);
3084 }
3085 
3086 /** @} */
std::map< int, job_t > job_map
Definition: remake.cpp:569
static void create_server()
Definition: remake.cpp:2542
std::string const & input
Definition: remake.cpp:811
static bool still_need_rebuild(std::string const &target)
Definition: remake.cpp:2032
int depth
Definition: remake.cpp:762
input_status next(std::string &)
Definition: remake.cpp:1369
int job_id
Job for which the built script called remake and spawned the client (negative for original clients)...
Definition: remake.cpp:589
static status_e run_script(int job_id, job_t const &job)
Definition: remake.cpp:2191
ref_ptr & operator=(ref_ptr const &p)
Definition: remake.cpp:480
status_e
Definition: remake.cpp:509
static void complete_request(client_t &client, bool success)
Definition: remake.cpp:2370
static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
Definition: remake.cpp:938
static generator * get_function(input_generator const &, std::string const &)
Definition: remake.cpp:1399
std::map< std::string, status_t > status_map
Definition: remake.cpp:529
static void find_rule(job_t &job, std::string const &target)
Definition: remake.cpp:1865
static void save_dependencies()
Definition: remake.cpp:1472
static void server_mode(std::string const &remakefile, string_list const &targets)
Definition: remake.cpp:2817
std::string pre
Definition: remake.cpp:1354
static void init_prefix_dir()
Definition: remake.cpp:882
#define DEBUG_open
Definition: remake.cpp:801
static char * socket_name
Definition: remake.cpp:695
static job_map jobs
Definition: remake.cpp:629
static bool propagate_vars
Definition: remake.cpp:736
std::ostream & operator()(bool o)
Definition: remake.cpp:774
static variable_map variables
Definition: remake.cpp:604
std::list< std::string > string_list
Definition: remake.cpp:456
string_list targets
Definition: remake.cpp:498
variable_map vars
Variables set on request.
Definition: remake.cpp:593
addprefix_generator(input_generator const &, bool &)
Definition: remake.cpp:1303
static int max_active_jobs
Definition: remake.cpp:646
escape_string(std::string const &s)
Definition: remake.cpp:812
T & operator*() const
Definition: remake.cpp:488
bool delayed
Whether it is a dependency client and a script has to be started on request completion.
Definition: remake.cpp:594
Target is being rebuilt.
Definition: remake.cpp:514
log()
Definition: remake.cpp:763
input_status next(std::string &)
Definition: remake.cpp:1231
std::string name
Definition: remake.cpp:1178
std::set< std::string > string_set
Definition: remake.cpp:458
static rule_map specific_rules
Definition: remake.cpp:624
static int waiting_jobs
Definition: remake.cpp:673
string_list wdeps
Like deps, except that they are not registered as dependencies.
Definition: remake.cpp:549
Target is up-to-date.
Definition: remake.cpp:511
static time_t now
Definition: remake.cpp:716
ref_ptr(ref_ptr const &p)
Definition: remake.cpp:478
static status_t const & get_status(std::string const &target)
Definition: remake.cpp:1922
std::map< std::string, string_list > variable_map
Definition: remake.cpp:504
Definition: remake.cpp:759
static void load_rule(std::istream &in, std::string const &first)
Definition: remake.cpp:1586
static std::string first_target
Definition: remake.cpp:701
std::list< rule_t > rule_list
Definition: remake.cpp:554
static void server_loop()
Definition: remake.cpp:2756
static bool build_failure
Definition: remake.cpp:689
static client_list clients
Definition: remake.cpp:640
static std::string read_word(std::istream &in, bool detect_equal=true)
Definition: remake.cpp:1105
string_list deps
Dependencies used for an implicit call to remake at the start of the script.
Definition: remake.cpp:548
ref_ptr(T const &t)
Definition: remake.cpp:477
string_list::const_iterator sufi
Definition: remake.cpp:1352
bool append
Definition: remake.cpp:536
static bool skip_eol(std::istream &in, bool multi=false)
Definition: remake.cpp:1030
static bool read_words(input_generator &in, string_list &res)
Definition: remake.cpp:1271
static void register_scripted_rule(rule_t const &rule)
Definition: remake.cpp:1556
string_set deps
Definition: remake.cpp:499
static void init_working_dir()
Definition: remake.cpp:860
static std::ostream & operator<<(std::ostream &out, escape_string const &se)
Definition: remake.cpp:819
static bool keep_going
Definition: remake.cpp:652
static pid_job_map job_pids
Definition: remake.cpp:634
char ** environ
static std::string normalize_abs(std::string const &s, std::string const &p)
Definition: remake.cpp:916
assign_map assigns
Assignment of variables.
Definition: remake.cpp:550
static bool echo_scripts
Definition: remake.cpp:711
string_list::const_iterator vend
Definition: remake.cpp:1179
input_generator(std::istream &i, variable_map const *lv, bool e=false)
Definition: remake.cpp:1223
Definition: remake.cpp:1161
static status_map status
Definition: remake.cpp:614
input_generator gen
Definition: remake.cpp:1350
static std::string working_dir
Definition: remake.cpp:721
static std::string prefix_dir
Definition: remake.cpp:726
bool open
Definition: remake.cpp:761
static bool obsolete_targets
Definition: remake.cpp:741
std::ostream & operator()()
Definition: remake.cpp:766
static void sigint_handler(int)
Definition: remake.cpp:751
std::map< std::string, ref_ptr< dependency_t > > dependency_map
Definition: remake.cpp:502
variable_map vars
Values of local variables.
Definition: remake.cpp:566
static volatile sig_atomic_t got_SIGCHLD
Definition: remake.cpp:744
input_status
Definition: remake.cpp:1157
static int running_jobs
Definition: remake.cpp:665
static void update_status(std::string const &target)
Definition: remake.cpp:1999
variable_generator(std::string const &, variable_map const *)
Definition: remake.cpp:1184
Target is missing or obsolete.
Definition: remake.cpp:512
std::string stem
Pattern used to instantiate the generic rule, if any.
Definition: remake.cpp:565
std::istream & in
Definition: remake.cpp:1219
string_list::const_iterator prei
Definition: remake.cpp:1296
static bool handle_clients()
Definition: remake.cpp:2427
static void accept_client()
Definition: remake.cpp:2622
string_list::const_iterator vcur
Definition: remake.cpp:1179
int socket_t
Definition: remake.cpp:447
bool still_open
Definition: remake.cpp:790
client_t()
Definition: remake.cpp:595
static bool show_targets
Definition: remake.cpp:706
addsuffix_generator(input_generator const &, bool &)
Definition: remake.cpp:1359
static void merge_rule(rule_t &dest, rule_t const &src)
Definition: remake.cpp:1791
Build failed for target.
Definition: remake.cpp:517
static log debug
Definition: remake.cpp:786
string_list targets
Files produced by this rule.
Definition: remake.cpp:547
static bool has_free_slots()
Definition: remake.cpp:2404
#define DEBUG
Definition: remake.cpp:800
Target has an obsolete dependency.
Definition: remake.cpp:513
std::string script
Shell script for building the targets.
Definition: remake.cpp:551
content(T const &t)
Definition: remake.cpp:473
Target was successfully rebuilt.
Definition: remake.cpp:516
static bool changed_prefix_dir
Definition: remake.cpp:731
static void skip_spaces(std::istream &in)
Definition: remake.cpp:1009
input_status next(std::string &)
Definition: remake.cpp:1203
#define DEBUG_close
Definition: remake.cpp:802
string_set running
Targets being built.
Definition: remake.cpp:592
~ref_ptr()
Definition: remake.cpp:479
static void normalize_list(string_list &l, std::string const &w, std::string const &p)
Definition: remake.cpp:989
std::map< std::string, ref_ptr< rule_t > > rule_map
Definition: remake.cpp:556
static void skip_empty(std::istream &in)
Definition: remake.cpp:1019
static void find_generic_rule(job_t &job, std::string const &target)
Definition: remake.cpp:1830
socket_t socket
Socket used to reply to the client (invalid for pseudo clients).
Definition: remake.cpp:588
content * ptr
Definition: remake.cpp:475
ref_ptr()
Definition: remake.cpp:476
static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
Definition: remake.cpp:1814
generator * nested
Definition: remake.cpp:1220
static socket_t socket_fd
Definition: remake.cpp:684
rule_t rule
Original rule.
Definition: remake.cpp:564
string_list value
Definition: remake.cpp:537
static int expect_token(std::istream &in, int mask)
Definition: remake.cpp:1059
input_generator gen
Definition: remake.cpp:1294
string_list pending
Targets not yet started.
Definition: remake.cpp:591
bool active
Definition: remake.cpp:761
virtual ~generator()
Definition: remake.cpp:1169
static void complete_job(int job_id, bool success, bool started=true)
Definition: remake.cpp:2066
static void usage(int exit_status)
Definition: remake.cpp:2957
static void client_mode(char *socket_name, string_list const &targets)
Definition: remake.cpp:2865
static int job_counter
Definition: remake.cpp:679
static void finalize_job(pid_t pid, bool res)
Definition: remake.cpp:2741
std::list< client_t > client_list
Definition: remake.cpp:598
std::map< pid_t, int > pid_job_map
Definition: remake.cpp:571
std::map< std::string, assign_t > assign_map
Definition: remake.cpp:540
bool failed
Whether some targets failed in mode -k.
Definition: remake.cpp:590
static status_e start(std::string const &target, client_list::iterator &current)
Definition: remake.cpp:2312
static void load_dependencies(std::istream &in)
Definition: remake.cpp:1422
static void load_rules(std::string const &remakefile)
Definition: remake.cpp:1720
static void register_transparent_rule(rule_t const &rule, string_list const &targets)
Definition: remake.cpp:1513
time_t last
Last-modified date.
Definition: remake.cpp:526
input_status next(std::string &)
Definition: remake.cpp:1313
status_e status
Actual status.
Definition: remake.cpp:525
static rule_list generic_rules
Definition: remake.cpp:619
int main(int argc, char *argv[])
Definition: remake.cpp:2984
virtual input_status next(std::string &)=0
static dependency_map dependencies
Definition: remake.cpp:609
std::string suf
Definition: remake.cpp:1298
T * operator->() const
Definition: remake.cpp:493
static std::string prepare_script(job_t const &job)
Definition: remake.cpp:2108
Static prerequisites are being rebuilt.
Definition: remake.cpp:515
static void sigchld_handler(int)
Definition: remake.cpp:746
variable_map const * local_variables
Definition: remake.cpp:1221
bool earliest_exit
Definition: remake.cpp:1222
string_list pre
Definition: remake.cpp:1295
string_list suf
Definition: remake.cpp:1351