[pph] Force token location for replayed macro definitions

[Gabriel Charette, 2011-08-16 20:38:50]

This patch follows issue4907044.

Replayed definitions were given new source locations in the lexer. This would
shift all of the locations assigned later and since this replay is absent in the
non-pph, we get different source_locations for the tokens following the replay
in pph.

This improves the quality of the pph line_table (closer and closer to the
non-pph line_table, but not perfect yet).

Tested with bootstrap and pph regression testing.

Gab

2011-08-16  Gabriel Charette  <gchare@google.com>

	gcc/cp/ChangeLog.pph
	* pph-streamer-in.c (pph_read_file_1): Set location of replayed
	tokens to column 0 of the line the pph file was included on.

	libcpp/ChangeLog.pph
	* include/symtab.h (line-map.h): Add dependency.
	* symtab.c (cpp_lt_replay): Add LOC parameter and force locations
	of tokens replayed to *LOC if non-null.

diff --git a/gcc/cp/pph-streamer-in.c b/gcc/cp/pph-streamer-in.c
index f87e6a5..8d25de8 100644
--- a/gcc/cp/pph-streamer-in.c
+++ b/gcc/cp/pph-streamer-in.c
@@ -1485,6 +1485,7 @@ pph_read_file_1 (pph_stream *stream)
   tree t, file_keyed_classes, file_static_aggregates;
   unsigned i;
   VEC(tree,gc) *file_unemitted_tinfo_decls;
+  source_location cpp_token_replay_loc;
 
   if (flag_pph_debug >= 1)
     fprintf (pph_logfile, "PPH: Reading %s\n", stream->name);
@@ -1502,8 +1503,13 @@ pph_read_file_1 (pph_stream *stream)
     report_validation_error (stream->name, bad_use->ident_str, cur_def,
                              bad_use->before_str, bad_use->after_str);
 
-  /* Re-instantiate all the pre-processor symbols defined by STREAM.  */
-  cpp_lt_replay (parse_in, &idents_used);
+  /* Re-instantiate all the pre-processor symbols defined by STREAM.  Force
+     their source_location to column 0 of the line the include occured on,
+     this avoids shifting all of the line_table's location as we would by
adding
+     locations which wouldn't be there in the non-pph compile; thus working
+     towards an identical line_table in pph and non-pph.  */
+  cpp_token_replay_loc = linemap_position_for_column (line_table, 0);
+  cpp_lt_replay (parse_in, &idents_used, &cpp_token_replay_loc);
 
   /* Read in STREAM's line table and merge it in the current line table.  */
   pph_in_and_merge_line_table (stream, line_table);
diff --git a/libcpp/include/symtab.h b/libcpp/include/symtab.h
index b5d6492..c39fa26 100644
--- a/libcpp/include/symtab.h
+++ b/libcpp/include/symtab.h
@@ -20,6 +20,7 @@ along with this program; see the file COPYING3.  If not see
 #define LIBCPP_SYMTAB_H
 
 #include "obstack.h"
+#include "line-map.h"
 
 #ifndef GTY
 #define GTY(x) /* nothing */
@@ -167,7 +168,8 @@ cpp_lt_verify (struct cpp_reader *reader, cpp_idents_used*
identifiers,
 /* Replay the macro definitions captured by the table of IDENTIFIERS
    into the READER state.  */
 void
-cpp_lt_replay (struct cpp_reader *reader, cpp_idents_used* identifiers);
+cpp_lt_replay (struct cpp_reader *reader, cpp_idents_used* identifiers,
+               source_location *loc);
 
 /* Destroy IDENTIFIERS captured.  */
 void
diff --git a/libcpp/symtab.c b/libcpp/symtab.c
index aad7277..b2e72f1 100644
--- a/libcpp/symtab.c
+++ b/libcpp/symtab.c
@@ -818,10 +818,12 @@ cpp_lt_define_syntax (char *needed, const char *ident,
const char *given)
 }
 
 /* Replay the macro definitions captured by the table of IDENTIFIERS
-   into the READER state.  */
+   into the READER state.  If LOC is non-null, assign *LOC as the
+   source_location to all macro definitions replayed.  */
 
 void
-cpp_lt_replay (cpp_reader *reader, cpp_idents_used* identifiers)
+cpp_lt_replay (cpp_reader *reader, cpp_idents_used* identifiers,
+               source_location *loc)
 {
   unsigned int i;
   unsigned int num_entries = identifiers->num_entries;
@@ -832,6 +834,9 @@ cpp_lt_replay (cpp_reader *reader, cpp_idents_used*
identifiers)
   /* Prevent the lexer from invalidating the tokens we've read so far.  */
   reader->keep_tokens++;
 
+  if (loc)
+    cpp_force_token_locations (reader, loc);
+
   for (i = 0; i < num_entries; ++i)
     {
       cpp_ident_use *entry = entries + i;
@@ -865,6 +870,9 @@ cpp_lt_replay (cpp_reader *reader, cpp_idents_used*
identifiers)
 
   reader->keep_tokens--;
 
+  if (loc)
+    cpp_stop_forcing_token_locations (reader);
+
   free (buffer);
 }
 

--
This patch is available for review at http://codereview.appspot.com/4905050

[Diego Novillo, 2011-08-16 20:40:37]

On 08/16/2011 04:38 PM, Gabriel Charette wrote:

> 2011-08-16  Gabriel Charette<gchare@google.com>
>
> 	gcc/cp/ChangeLog.pph
> 	* pph-streamer-in.c (pph_read_file_1): Set location of replayed
> 	tokens to column 0 of the line the pph file was included on.
>
> 	libcpp/ChangeLog.pph
> 	* include/symtab.h (line-map.h): Add dependency.
> 	* symtab.c (cpp_lt_replay): Add LOC parameter and force locations
> 	of tokens replayed to *LOC if non-null.

OK.


Diego.