Issue 4303049: [pph] AST streaming (1/4) (Closed)
|
Created:
13 years, 1 month ago by Diego Novillo Modified:
13 years, 1 month ago Reviewers:
CC:
Lawrence Crowl Visibility:
Public. |
Patch Set 1 #Patch Set 2 : [pph] AST streaming (2/4) #Patch Set 3 : [pph] AST streaming (3/4) #Patch Set 4 : [pph] AST streaming (4/4) #MessagesTotal messages: 4
This set of 4 patches sets up the basic streaming facility for PPH. The idea is to re-use common code in LTO streaming and add streaming hooks for dealing with front end specific data. As we discussed earlier today (http://gcc.gnu.org/ml/gcc/2011-03/msg00354.html), the front end needs to handle more tree codes that we do not care about in gimple. Furthermore, we are interested in removing some of the tree codes that gimple is forced to handle. So, the actual pickling of trees will eventually move into language specific files and we will add streaming hooks to call from common code. These patches are actually a single unit, but I split them in 4 parts for clarity: 1- Adds the inevitable tracer flag (-fpph-tracer) to show what the writer and the reader think they are doing. 2- Adds the main streaming support. For now PPH files are very simple: - Identification string - Version (major, minor, patchlevel) - Size of string table. - String table. - Symbol table. This is the extent of the data that we are currently streaming. PPH streaming works similarly to FILE I/O: pph_stream_open(): Opens a pph file by setting up all the data structures and tables needed by LTO routines. pph_output_*/pph_input_*: Wrappers around LTO streaming routines. pph_stream_close(): Writes out all the streams built by the pph_output_* calls. Above these pph streaming routines we are implementing the high-level logic for saving and restoring the parser state. This patch works now for PTH (pre-tokenized headers). There is also some early saving of parsed trees, but it is still very incomplete. 3- Exports some LTO functions that were private to the LTO streamer. 4- Hacks around some checks and assumptions that LTO makes about gimple. These are the ones that I will need to either add hooks or factor out somehow. For now I marked them with FIXME notes. I want to get most/all of them before I decide how to factor things out. Lawrence, we have some regressions in pph.exp because we can't handle template decls. I'll be fixing that in the next few days. Tested on x86_64. Diego. c-family/ChangeLog.pph * c-family/c.opt (fpph-hdr, fpph-map): Re-order. (fpph-tracer): New. diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt index fd4406f..211586a 100644 --- a/gcc/c-family/c.opt +++ b/gcc/c-family/c.opt @@ -925,14 +925,6 @@ fpph C++ Var(flag_pph) -fpph Enable pre-parsed header (PPH) support -fpph-hdr= -C++ ObjC++ Joined MissingArgError(missing filename after %qs) --fpph-hdr=<base-name> A mapping from <base-name>.h to <base-name>.pph - -fpph-map= -C++ ObjC++ Joined MissingArgError(missing filename after %qs) --fpph-map=<file-name> A file of mappings from #include to PPH file - fpph-debug= C++ Joined RejectNegative UInteger Var(flag_pph_debug) -fpph-debug=N Enable debugging output at level N from PPH support @@ -945,14 +937,26 @@ fpph-fmt= C++ Joined RejectNegative UInteger Var(flag_pph_fmt) -fpph-fmt=N Output format is (0) normal (1) pretty summary (2) dump +fpph-hdr= +C++ ObjC++ Joined MissingArgError(missing filename after %qs) +-fpph-hdr=<base-name> A mapping from <base-name>.h to <base-name>.pph + fpph-logfile= C++ Joined RejectNegative Var(flag_pph_logfile) -fpph-logfile=<file-name> Emit PPH debug information to <file-name> +fpph-map= +C++ ObjC++ Joined MissingArgError(missing filename after %qs) +-fpph-map=<file-name> A file of mappings from #include to PPH file + fpph-stats C++ Var(flag_pph_stats) -fpph-stats Enable statistics gathering for PPH +fpph-tracer +C++ Var(flag_pph_tracer) +-fpph-tracer Enable tracing of PPH streaming operations + fpreprocessed C ObjC C++ ObjC++ Treat the input file as already preprocessed -- This patch is available for review at http://codereview.appspot.com/4303049
Sign in to reply to this message.
Main streaming support. See http://gcc.gnu.org/ml/gcc-patches/2011-03/msg01556.html for details. cp/ChangeLog.pph 2011-03-23 Diego Novillo <dnovillo@google.com> * Make-lang.in (cp/pph-streamer.o): Add dependencies on tree-pretty-print.h, CXX_PPH_H, TREE_PASS_H, version.h and cppbuiltin.h * pph-streamer.c: Include tree-pretty-print.h, pph.h, tree-pass.h, version.h and cppbuiltin.h. (pph_get_section_data): New. (pph_free_section_data): New. (pph_stream_init_read): Rename from pph_file_read. Initialize LTO streaming data structures. Read pph file in memory. (pph_stream_init_write): Factor out of ... (pph_stream_open): ... here. (pph_stream_write_header): New. (pph_stream_write_body): Factor out of ... (pph_stream_close): ... here. Call pph_stream_write_header. (enum pph_trace_type): Declare. (pph_stream_trace): New. (pph_stream_trace_tree): New. (pph_stream_trace_uint): New. (pph_stream_trace_bytes): New. (pph_stream_trace_string): New. (pph_stream_trace_string_with_length): New. * pph-streamer.h (PPH_NUM_SECTIONS): Define. (PPH_USE_FILE_IO): Remove. Update all users. (pph_id_str): Define. (pph_file_header): Declare. (pph_stream): Add fields pph_sections, file_data and file_size. (pph_stream_open): Change second argument to const char *. Update all users. (pph_stream_trace_tree): Declare. (pph_stream_trace_uint): Declare. (pph_stream_trace_bytes): Declare. (pph_stream_trace_string): Declare. (pph_stream_trace_string_with_length): Declare. (pph_output_tree): Call pph_stream_trace_tree if flag_pph_tracer is enabled. (pph_input_tree): Likewise. (pph_output_uint): Call pph_stream_trace_uint if flag_pph_tracer is enabled. (pph_input_uint): Likewise. (pph_output_bytes): Call pph_stream_trace_bytes if flag_pph_tracer is enabled. (pph_input_bytes): Likewise. (pph_output_string): Call pph_stream_trace_string if flag_pph_tracer is enabled. (pph_input_string): Likewise. (pph_output_string_with_length): Call pph_stream_trace_string_with_length if flag_pph_tracer is enabled. * pph.c (pph_print_macro_defs_before): Call pph_output_string. (pph_print_macro_defs_after): Likewise. (pth_load_token_value): Do not free the read string. (pth_load_identifiers): Likewise. (pth_load_include): Likewise. (pth_load_image): Call pph_input_bytes to read the header. (write_pph_print): Call pph_output_tree. (write_pph_file_object): Remove fprintf() call. (pth_include_handler): Remove FIXME marker. (pph_include_handler): Tidy. diff --git a/gcc/cp/Make-lang.in b/gcc/cp/Make-lang.in index 3a52e1f..bcb2ba1 100644 --- a/gcc/cp/Make-lang.in +++ b/gcc/cp/Make-lang.in @@ -324,12 +324,10 @@ cp/parser.o: cp/parser.c $(CXX_TREE_H) $(TM_H) $(DIAGNOSTIC_CORE_H) \ $(CXX_PPH_H) cp/cp-gimplify.o: cp/cp-gimplify.c $(CXX_TREE_H) $(C_COMMON_H) \ $(TM_H) coretypes.h pointer-set.h tree-iterator.h - cp/name-lookup.o: cp/name-lookup.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \ $(TM_H) $(CXX_TREE_H) $(TIMEVAR_H) gt-cp-name-lookup.h \ $(DIAGNOSTIC_CORE_H) $(FLAGS_H) debug.h tree-pretty-print.h \ $(CXX_PPH_H) - cp/cxx-pretty-print.o: cp/cxx-pretty-print.c $(CXX_PRETTY_PRINT_H) \ $(CONFIG_H) $(SYSTEM_H) $(TM_H) coretypes.h $(CXX_TREE_H) tree-pretty-print.h cp/pph.o: cp/pph.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(CPPLIB_H) \ @@ -337,4 +335,5 @@ cp/pph.o: cp/pph.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(CPPLIB_H) \ fixed-value.h $(TREE_PASS_H) $(TREE_INLINE_H) tree-pretty-print.h \ $(CXX_PARSER_H) $(CXX_PPH_H) $(CXX_PPH_STREAMER_H) cp/pph-streamer.o: cp/pph-streamer.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \ - $(TREE_H) $(LTO_STREAMER_H) $(CXX_PPH_STREAMER_H) + $(TREE_H) tree-pretty-print.h $(LTO_STREAMER_H) $(CXX_PPH_STREAMER_H) \ + $(CXX_PPH_H) $(TREE_PASS_H) version.h cppbuiltin.h diff --git a/gcc/cp/pph-streamer.c b/gcc/cp/pph-streamer.c index c170f1f..cca42e6 100644 --- a/gcc/cp/pph-streamer.c +++ b/gcc/cp/pph-streamer.c @@ -23,68 +23,147 @@ along with GCC; see the file COPYING3. If not see #include "coretypes.h" #include "tree.h" #include "langhooks.h" +#include "tree-pretty-print.h" #include "lto-streamer.h" #include "pph-streamer.h" - -#if !defined PPH_USE_FILE_IO +#include "pph.h" +#include "tree-pass.h" +#include "version.h" +#include "cppbuiltin.h" + +/* FIXME pph. This holds the FILE handle for the current PPH file + that we are writing. It is necessary because the LTO callbacks do + not allow passing a FILE handle to them. */ static FILE *current_pph_file = NULL; -#endif + + +/* Get the section with name NAME and type SECTION_TYPE from FILE_DATA. + Return a pointer to the start of the section contents and store + the length of the section in *LEN_P. + + FIXME pph, this does not currently handle multiple sections. It + assumes that the file has exactly one section. */ + +static const char * +pph_get_section_data (struct lto_file_decl_data *file_data, + enum lto_section_type section_type ATTRIBUTE_UNUSED, + const char *name ATTRIBUTE_UNUSED, + size_t *len) +{ + /* FIXME pph - Stop abusing lto_file_decl_data fields. */ + const pph_stream *stream = (const pph_stream *) file_data->file_name; + *len = stream->file_size - sizeof (pph_file_header); + return (const char *) stream->file_data + sizeof (pph_file_header); +} + + +/* Free the section data from FILE_DATA of SECTION_TYPE and NAME that + starts at OFFSET and has LEN bytes. */ + +static void +pph_free_section_data (struct lto_file_decl_data *file_data, + enum lto_section_type section_type ATTRIBUTE_UNUSED, + const char *name ATTRIBUTE_UNUSED, + const char *offset ATTRIBUTE_UNUSED, + size_t len ATTRIBUTE_UNUSED) +{ + /* FIXME pph - Stop abusing lto_file_decl_data fields. */ + const pph_stream *stream = (const pph_stream *) file_data->file_name; + free (stream->file_data); +} + /* Read into memory the contents of the file in STREAM. Initialize internal tables and data structures needed to re-construct the ASTs in the file. */ static void -pph_file_read (pph_stream *stream) +pph_stream_init_read (pph_stream *stream) { -#if defined PPH_USE_FILE_IO - stream->file = fopen (stream->name, "rb"); -#else - void *data; struct stat st; - size_t bytes_read, data_size; - - /* Read STREAM->NAME into the memory buffer DATA. */ - stat (stream->name, &st); - data_size = (size_t) st.st_size; - data = XCNEWVEC (char, data_size); - stream->file = fopen (stream->name, "rb"); - gcc_assert (stream->file); - bytes_read = fread (data, 1, data_size, stream->file); - gcc_assert (bytes_read == data_size); - fclose (stream->file); + size_t i, bytes_read, strtab_size, body_size; + int retcode; + pph_file_header *header; + const char *strtab, *body; + + /* Read STREAM->NAME into the memory buffer STREAM->FILE_DATA. + FIXME pph, we are reading the whole file at once. This seems + wasteful. */ + retcode = fstat (fileno (stream->file), &st); + gcc_assert (retcode == 0); + stream->file_size = (size_t) st.st_size; + stream->file_data = XCNEWVEC (char, stream->file_size); + bytes_read = fread (stream->file_data, 1, stream->file_size, stream->file); + gcc_assert (bytes_read == stream->file_size); + + /* Set LTO callbacks to read the PPH file. */ + stream->pph_sections = XCNEWVEC (struct lto_file_decl_data *, + PPH_NUM_SECTIONS); + for (i = 0; i < PPH_NUM_SECTIONS; i++) + { + stream->pph_sections[i] = XCNEW (struct lto_file_decl_data); + /* FIXME pph - Stop abusing fields in lto_file_decl_data. */ + stream->pph_sections[i]->file_name = (const char *) stream; + } + + lto_set_in_hooks (stream->pph_sections, pph_get_section_data, + pph_free_section_data); + header = (pph_file_header *) stream->file_data; + strtab = (const char *) header + sizeof (pph_file_header); + strtab_size = header->strtab_size; + body = strtab + strtab_size; + gcc_assert (stream->file_size >= strtab_size + sizeof (pph_file_header)); + body_size = stream->file_size - strtab_size - sizeof (pph_file_header); + + /* Create an input block structure pointing right after the string + table. */ stream->ib = XCNEW (struct lto_input_block); - LTO_INIT_INPUT_BLOCK_PTR (stream->ib, (const char *) data, 0, data_size); - stream->data_in = lto_data_in_create (NULL, NULL, 0, NULL); -#endif + LTO_INIT_INPUT_BLOCK_PTR (stream->ib, body, 0, body_size); + stream->data_in = lto_data_in_create (stream->pph_sections[0], strtab, + strtab_size, NULL); +} + + +/* Initialize buffers and tables in STREAM for writing. */ + +static void +pph_stream_init_write (pph_stream *stream) +{ + stream->out_state = lto_new_out_decl_state (); + lto_push_out_decl_state (stream->out_state); + stream->decl_state_stream = XCNEW (struct lto_output_stream); + stream->ob = create_output_block (LTO_section_decls); } -/* Create a new PPH stream to be stored on the file called NAME. If - TO_READ_P is true, the file is open for reading. */ +/* Create a new PPH stream to be stored on the file called NAME. + MODE is passed to fopen directly. */ pph_stream * -pph_stream_open (const char *name, bool to_read_p) +pph_stream_open (const char *name, const char *mode) { - pph_stream *stream = XCNEW (pph_stream); - stream->name = xstrdup (name); - if (!to_read_p) + pph_stream *stream; + FILE *f; + + stream = NULL; + f = fopen (name, mode); + if (f) { - stream->file = fopen (name, "wb"); - stream->out_state = lto_new_out_decl_state (); - lto_push_out_decl_state (stream->out_state); - stream->decl_state_stream = XCNEW (struct lto_output_stream); - stream->ob = create_output_block (LTO_section_decls); + stream = XCNEW (pph_stream); + stream->file = f; + stream->name = xstrdup (name); + stream->write_p = (strchr (mode, 'w') != NULL); + if (stream->write_p) + pph_stream_init_write (stream); + else + pph_stream_init_read (stream); } - else - pph_file_read (stream); return stream; } -#if !defined PPH_USE_FILE_IO /* Callback for lang_hooks.lto.begin_section. Open file NAME. */ static void @@ -112,31 +191,44 @@ static void pph_stream_end_section (void) { } -#endif -/* Close PPH stream STREAM. Write all the ASTs to disk and deallocate - all memory used by it. */ +/* Write the header for the PPH file represented by STREAM. */ -void -pph_stream_close (pph_stream *stream) +static void +pph_stream_write_header (pph_stream *stream) { -#if defined PPH_USE_FILE_IO - fclose (stream->file); -#else - gcc_assert (current_pph_file == NULL); - current_pph_file = stream->file; + pph_file_header header; + struct lto_output_stream header_stream; + int major, minor, patchlevel; + + /* Collect version information. */ + parse_basever (&major, &minor, &patchlevel); + gcc_assert (major == (char) major); + gcc_assert (minor == (char) minor); + gcc_assert (patchlevel == (char) patchlevel); + + /* Write the header for the PPH file. */ + memset (&header, 0, sizeof (header)); + strcpy (header.id_str, pph_id_str); + header.major_version = (char) major; + header.minor_version = (char) minor; + header.patchlevel = (char) patchlevel; + header.strtab_size = stream->ob->string_stream->total_size; + + memset (&header_stream, 0, sizeof (header_stream)); + lto_output_data_stream (&header_stream, &header, sizeof (header)); + lto_write_stream (&header_stream); +} - /* Redirect the LTO basic I/O langhooks. */ - lang_hooks.lto.begin_section = pph_stream_begin_section; - lang_hooks.lto.append_data = pph_stream_write; - lang_hooks.lto.end_section = pph_stream_end_section; - /* Write the state buffers built by pph_stream_output() calls. */ - lto_begin_section (stream->name, false); +/* Write the body of the PPH file represented by STREAM. */ - /* Make string 0 be a NULL string. */ - lto_output_1_stream (stream->ob->string_stream, 0); +static void +pph_stream_write_body (pph_stream *stream) +{ + /* Write the string table. */ + lto_write_stream (stream->ob->string_stream); /* Write out the physical representation for every AST in all the streams in STREAM->OUT_STATE. */ @@ -148,10 +240,148 @@ pph_stream_close (pph_stream *stream) /* Finally, physically write all the streams. */ lto_write_stream (stream->ob->main_stream); - lto_write_stream (stream->ob->string_stream); +} + + +/* Close PPH stream STREAM. Write all the ASTs to disk and deallocate + all memory used by it. */ + +void +pph_stream_close (pph_stream *stream) +{ + if (stream->write_p) + { + gcc_assert (current_pph_file == NULL); + current_pph_file = stream->file; + + /* Redirect the LTO basic I/O langhooks. */ + lang_hooks.lto.begin_section = pph_stream_begin_section; + lang_hooks.lto.append_data = pph_stream_write; + lang_hooks.lto.end_section = pph_stream_end_section; + + /* Write the state buffers built by pph_output_*() calls. */ + lto_begin_section (stream->name, false); + pph_stream_write_header (stream); + pph_stream_write_body (stream); + lto_end_section (); + } - lto_end_section (); fclose (stream->file); - current_pph_file = NULL; -#endif + stream->file = current_pph_file = NULL; +} + + +/* Data types supported by the PPH tracer. */ +enum pph_trace_type +{ + PPH_TRACE_TREE, + PPH_TRACE_UINT, + PPH_TRACE_BYTES, + PPH_TRACE_STRING +}; + +/* Print tracing information for STREAM on pph_logfile. DATA is the + memory area to display, SIZE is the number of bytes to print, TYPE + is the kind of data to print. */ + +static void +pph_stream_trace (pph_stream *stream, const void *data, unsigned int nbytes, + enum pph_trace_type type) +{ + const char *op = (stream->write_p) ? "write" : "read"; + const char *type_s[] = { "tree", "uint", "bytes", "string" }; + + fprintf (pph_logfile, "*** %s: op=%s, type=%s, size=%u, value=", + stream->name, op, type_s[type], (unsigned) nbytes); + + switch (type) + { + case PPH_TRACE_TREE: + { + const_tree t = (const_tree) data; + print_generic_expr (pph_logfile, CONST_CAST (union tree_node *, t), + TDF_SLIM); + } + break; + + case PPH_TRACE_UINT: + { + unsigned int val = *((const unsigned int *) data); + fprintf (pph_logfile, "%u (0x%x)", val, val); + } + break; + + case PPH_TRACE_BYTES: + { + size_t i; + const char *buffer = (const char *) data; + for (i = 0; i < MIN (nbytes, 100); i++) + { + if (ISPRINT (buffer[i])) + fprintf (pph_logfile, "%c", buffer[i]); + else + fprintf (pph_logfile, "[0x%02x]", (unsigned int) buffer[i]); + } + } + break; + + case PPH_TRACE_STRING: + if (data) + fprintf (pph_logfile, "%.*s", (int) nbytes, (const char *) data); + else + fprintf (pph_logfile, "<nil>"); + break; + + default: + gcc_unreachable (); + } + + fputc ('\n', pph_logfile); +} + + +/* Show tracing information for T on STREAM. */ + +void +pph_stream_trace_tree (pph_stream *stream, tree t) +{ + pph_stream_trace (stream, t, tree_code_size (TREE_CODE (t)), PPH_TRACE_TREE); +} + + +/* Show tracing information for VAL on STREAM. */ + +void +pph_stream_trace_uint (pph_stream *stream, unsigned int val) +{ + pph_stream_trace (stream, &val, sizeof (val), PPH_TRACE_UINT); +} + + +/* Show tracing information for NBYTES bytes of memory area DATA on + STREAM. */ + +void +pph_stream_trace_bytes (pph_stream *stream, const void *data, size_t nbytes) +{ + pph_stream_trace (stream, data, nbytes, PPH_TRACE_BYTES); +} + + +/* Show tracing information for S on STREAM. */ + +void +pph_stream_trace_string (pph_stream *stream, const char *s) +{ + pph_stream_trace (stream, s, s ? strlen (s) : 0, PPH_TRACE_STRING); +} + + +/* Show tracing information for LEN bytes of S on STREAM. */ + +void +pph_stream_trace_string_with_length (pph_stream *stream, const char *s, + unsigned int len) +{ + pph_stream_trace (stream, s, len, PPH_TRACE_STRING); } diff --git a/gcc/cp/pph-streamer.h b/gcc/cp/pph-streamer.h index 1b04d65..afffe35 100644 --- a/gcc/cp/pph-streamer.h +++ b/gcc/cp/pph-streamer.h @@ -24,9 +24,32 @@ along with GCC; see the file COPYING3. If not see #include "lto-streamer.h" #include "tree.h" -/* FIXME pph - Workaround incomplete PPH streamer. Use regular FILE I/O. */ -#define PPH_USE_FILE_IO 1 +/* Number of sections in a PPH file. FIXME, currently only one section + is supported. To add more, it will also be necessary to handle + section names in pph_get_section_data and pph_free_section_data. */ +#define PPH_NUM_SECTIONS 1 +/* String to identify PPH files. Keep it to 7 characters, so it takes + exactly 8 bytes in the file. */ +static const char pph_id_str[] = "PPH0x42"; + +/* Structure of the header of a PPH file. */ +typedef struct pph_file_header { + /* Identification string. */ + char id_str[sizeof(pph_id_str)]; + + /* Version information. */ + char major_version; + char minor_version; + char patchlevel; + + /* Size of the string table in bytes. */ + unsigned int strtab_size; +} pph_file_header; + + +/* A PPH stream contains all the data and attributes needed to + write symbols, declarations and other parsing products to disk. */ typedef struct pph_stream { /* Path name of the PPH file. */ const char *name; @@ -45,16 +68,29 @@ typedef struct pph_stream { struct lto_input_block *ib; /* String tables and other descriptors used by the LTO reading - routines. */ + routines. NULL when the file is opened for reading. */ struct data_in *data_in; - /* Nonzero if the stream was open for writing. */ + /* Array of sections in the PPH file. */ + struct lto_file_decl_data **pph_sections; + + /* Buffer holding the file contents. FIXME pph, we are bringing + the whole file in memory at once. This seems wasteful. */ + char *file_data; + size_t file_size; + + /* Nonzero if the stream was opened for writing. */ unsigned int write_p : 1; } pph_stream; /* In pph-streamer.c. */ -pph_stream *pph_stream_open (const char *, bool); +pph_stream *pph_stream_open (const char *, const char *); void pph_stream_close (pph_stream *); +void pph_stream_trace_tree (pph_stream *, tree); +void pph_stream_trace_uint (pph_stream *, unsigned int); +void pph_stream_trace_bytes (pph_stream *, const void *, size_t); +void pph_stream_trace_string (pph_stream *, const char *); +void pph_stream_trace_string_with_length (pph_stream *, const char *, unsigned); /* Inline functions. */ @@ -63,52 +99,36 @@ void pph_stream_close (pph_stream *); static inline void pph_output_tree (pph_stream *stream ATTRIBUTE_UNUSED, tree t ATTRIBUTE_UNUSED) { -#if defined PPH_USE_FILE_IO - gcc_unreachable (); -#else + if (flag_pph_tracer) + pph_stream_trace_tree (stream, t); lto_output_tree (stream->ob, t, true); -#endif } /* Write a uint VALUE to STREAM. */ static inline void pph_output_uint (pph_stream *stream, unsigned int value) { -#if defined PPH_USE_FILE_IO - fwrite (&value, 1, sizeof (value), stream->file); -#else + if (flag_pph_tracer) + pph_stream_trace_uint (stream, value); lto_output_sleb128_stream (stream->ob->main_stream, value); -#endif } /* Write N bytes from P to STREAM. */ static inline void pph_output_bytes (pph_stream *stream, const void *p, size_t n) { -#if defined PPH_USE_FILE_IO - fwrite (p, 1, n, stream->file); -#else + if (flag_pph_tracer) + pph_stream_trace_bytes (stream, p, n); lto_output_data_stream (stream->ob->main_stream, p, n); -#endif } /* Write string STR to STREAM. */ static inline void pph_output_string (pph_stream *stream, const char *str) { -#if defined PPH_USE_FILE_IO - if (str == NULL) - pph_output_uint (stream, -1U); - else - { - unsigned length = strlen (str); - pph_output_uint (stream, length); - if (length > 0) - pph_output_bytes (stream, str, length); - } -#else lto_output_string (stream->ob, stream->ob->main_stream, str); -#endif + if (flag_pph_tracer) + pph_stream_trace_string (stream, str); } /* Write string STR of length LEN to STREAM. */ @@ -116,35 +136,29 @@ static inline void pph_output_string_with_length (pph_stream *stream, const char *str, unsigned int len) { -#if defined PPH_USE_FILE_IO - if (str == NULL) - pph_output_uint (stream, -1U); + if (str) + lto_output_string_with_length (stream->ob, stream->ob->main_stream, + str, len + 1); else { - pph_output_uint (stream, len); - if (len > 0) - pph_output_bytes (stream, str, len); + /* lto_output_string_with_length does not handle NULL strings, + but lto_output_string does. */ + pph_output_string (stream, NULL); } -#else - lto_output_string_with_length (stream->ob, stream->ob->main_stream, str, len); -#endif + + if (flag_pph_tracer) + pph_stream_trace_string_with_length (stream, str, len); } /* Read an unsigned HOST_WIDE_INT integer from STREAM. */ -static inline unsigned +static inline unsigned int pph_input_uint (pph_stream *stream) { -#if defined PPH_USE_FILE_IO - unsigned num; - size_t received; - received = fread (&num, sizeof num, 1, stream->file); - gcc_assert (received == 1); - return num; -#else HOST_WIDE_INT unsigned n = lto_input_uleb128 (stream->ib); gcc_assert (n == (unsigned) n); + if (flag_pph_tracer) + pph_stream_trace_uint (stream, n); return (unsigned) n; -#endif } /* Read N bytes from STREAM into P. The caller is responsible for @@ -152,12 +166,9 @@ pph_input_uint (pph_stream *stream) static inline void pph_input_bytes (pph_stream *stream, void *p, size_t n) { -#if defined PPH_USE_FILE_IO - size_t received = fread (p, 1, n, stream->file); - gcc_assert (received == n); -#else lto_input_data_block (stream->ib, p, n); -#endif + if (flag_pph_tracer) + pph_stream_trace_bytes (stream, p, n); } /* Read and return a string of up to MAX characters from STREAM. @@ -167,20 +178,10 @@ pph_input_bytes (pph_stream *stream, void *p, size_t n) static inline const char * pph_input_string (pph_stream *stream) { -#if defined PPH_USE_FILE_IO - char *buf = NULL; - unsigned len; - size_t received = fread (&len, sizeof len, 1, stream->file); - gcc_assert (received == 1); - if (len > 0 && len != -1U) - { - buf = XCNEWVEC (char, len + 1); - received = fread (buf, 1, len, stream->file); - } - return (const char *) buf; -#else - return lto_input_string (stream->data_in, stream->ib); -#endif + const char *s = lto_input_string (stream->data_in, stream->ib); + if (flag_pph_tracer) + pph_stream_trace_string (stream, s); + return s; } /* Load an AST from STREAM. Return the corresponding tree. */ @@ -188,11 +189,10 @@ pph_input_string (pph_stream *stream) static inline tree pph_input_tree (pph_stream *stream ATTRIBUTE_UNUSED) { -#if defined PPH_USE_FILE_IO - gcc_unreachable (); -#else - return lto_input_tree (stream->ib, stream->data_in); -#endif + tree t = lto_input_tree (stream->ib, stream->data_in); + if (flag_pph_tracer) + pph_stream_trace_tree (stream, t); + return t; } #endif /* GCC_CP_PPH_STREAMER_H */ diff --git a/gcc/cp/pph.c b/gcc/cp/pph.c index e0ddf24..5ec7916 100644 --- a/gcc/cp/pph.c +++ b/gcc/cp/pph.c @@ -197,18 +197,17 @@ pth_name_for (const char *name) } -/* Open an image file for path NAME. READ_P is true if the file should - be opened for reading. */ +/* Open an image file for path NAME. MODE is as in fopen. */ static pph_stream * -pth_file_for (const char *name, bool read_p) +pth_file_for (const char *name, const char *mode) { char *s; pph_stream *f; s = pth_name_for (name); - f = pph_stream_open (s, read_p); - if (f->file == NULL) + f = pph_stream_open (s, mode); + if (!f) fatal_error ("can%'t open token stream file %s: %m", s); free (s); @@ -462,9 +461,16 @@ pph_print_macro_defs_before (pph_stream *stream, cpp_idents_used *identifiers) const char *before = entry->before_str; if (before) - fprintf (stream->file, "#define %s%s\n", ident, before); + { + pph_output_string (stream, "#define "); + pph_output_string (stream, ident); + pph_output_string (stream, before); + } else - fprintf (stream->file, "#undef %s\n", ident); + { + pph_output_string (stream, "#undef "); + pph_output_string (stream, ident); + } } } @@ -487,9 +493,16 @@ pph_print_macro_defs_after (pph_stream *stream, cpp_idents_used *identifiers) if (before != after) { if (after && (!before || strcmp (after, before) != 0)) - fprintf (stream->file, "#define %s%s\n", ident, after); + { + pph_output_string (stream, "#define "); + pph_output_string (stream, ident); + pph_output_string (stream, after); + } else if (before) - fprintf (stream->file, "#undef %s\n", ident); + { + pph_output_string (stream, "#undef "); + pph_output_string (stream, ident); + } } } } @@ -864,7 +877,7 @@ pth_save_image (pth_image *image) /* Open the stream in append mode since we have already created it in pth_new_image. */ - stream = pth_file_for (image->fname, false); + stream = pth_file_for (image->fname, "wb"); /* Write a header to recognize the file later. */ pth_write_header (image, stream); @@ -984,7 +997,6 @@ pth_load_token_value (cp_token *token, pph_stream *f) case CPP_NAME: str = pph_input_string (f); token->u.value = get_identifier (str); - free (CONST_CAST (char *, str)); break; case CPP_KEYWORD: @@ -1005,7 +1017,6 @@ pth_load_token_value (cp_token *token, pph_stream *f) case CPP_STRING32: str = pph_input_string (f); token->u.value = build_string (strlen (str), str); - free (CONST_CAST (char *, str)); break; case CPP_PRAGMA: @@ -1051,7 +1062,6 @@ pth_load_identifiers (cpp_idents_used *identifiers, pph_stream *stream) identifiers->entries[j].ident_len = ident_len; identifiers->entries[j].ident_str = (const char *) obstack_copy0 (identifiers->strings, s, ident_len); - free (CONST_CAST (char *, s)); s = pph_input_string (stream); if (s) @@ -1060,7 +1070,6 @@ pth_load_identifiers (cpp_idents_used *identifiers, pph_stream *stream) identifiers->entries[j].before_len = before_len; identifiers->entries[j].before_str = (const char *) obstack_copy0 (identifiers->strings, s, before_len); - free (CONST_CAST (char *, s)); } else { @@ -1077,7 +1086,6 @@ pth_load_identifiers (cpp_idents_used *identifiers, pph_stream *stream) identifiers->entries[j].after_len = after_len; identifiers->entries[j].after_str = (const char *) obstack_copy0 (identifiers->strings, s, after_len); - free (CONST_CAST (char *, s)); } else { @@ -1162,7 +1170,6 @@ pth_load_include (pth_state *state, pth_image *image, cpp_reader *reader, s = pph_input_string (stream); include->image = pth_image_lookup (state, s, reader); - free (CONST_CAST (char *, s)); tmp = pph_input_uint (stream); include->itype = (enum include_type) tmp; @@ -1184,14 +1191,17 @@ pth_load_image (pth_state *state, pth_image *image, cpp_reader *reader) { pph_stream *stream; unsigned i, num; + char *buf; timevar_push (TV_PTH_LOAD); - stream = pth_file_for (image->fname, true); + stream = pth_file_for (image->fname, "rb"); /* Skip over the header, as we assume that it has already been validated by pth_have_valid_image_for. */ - fseek (stream->file, (long) pth_header_len (), SEEK_SET); + buf = XCNEWVEC (char, pth_header_len ()); + pph_input_bytes (stream, buf, pth_header_len ()); + free (buf); /* Read the include-hunk (IH) sequencing vector. */ num = pph_input_uint (stream); @@ -1258,7 +1268,8 @@ pth_have_valid_image_for (const char *fname, pth_image *image) goto invalid_img; /* If the file exists, check if it has a valid signature. */ - f = pph_stream_open (img_name, true); + f = pph_stream_open (img_name, "rb"); + gcc_assert (f); good_id = pth_id_str (); id = XCNEWVEC (char, strlen (good_id) + 1); @@ -1834,17 +1845,17 @@ pth_file_change (cpp_reader *reader, const struct line_map *map) typedef void (*write_pph_format)(pph_stream *stream, tree decl, int flags); +/* Forward declarations to break cyclic references. */ static void -write_pph_namespace (pph_stream *stream, tree decl, write_pph_format fmt, int flags); +write_pph_namespace (pph_stream *, tree, write_pph_format, int); /* Write symbol to PPH output file like C. */ static void -write_pph_print (pph_stream *stream, tree decl, int flags) +write_pph_print (pph_stream *stream, tree decl, int flags ATTRIBUTE_UNUSED) { - print_generic_decl (stream->file, decl, flags | TDF_VISDEF); - fprintf (stream->file, "\n"); + pph_output_tree (stream, decl); } @@ -1860,7 +1871,8 @@ write_pph_dump (pph_stream *stream, tree decl, int flags) /* Write symbol to PPH output file. */ static void -write_pph_symbol (pph_stream *stream, tree decl, write_pph_format fmt, int flags) +write_pph_symbol (pph_stream *stream, tree decl, write_pph_format fmt, + int flags) { if (TREE_CODE (decl) == NAMESPACE_DECL) write_pph_namespace (stream, decl, fmt, flags); @@ -1884,7 +1896,8 @@ write_pph_namespace_1 (declvisitor vtor, pph_stream *stream, tree decl, } static void -write_pph_namespace (pph_stream *stream, tree decl, write_pph_format fmt, int flags) +write_pph_namespace (pph_stream *stream, tree decl, write_pph_format fmt, + int flags) { struct cp_binding_level *level = NAMESPACE_LEVEL (decl); decl = level->namespaces; @@ -1903,8 +1916,6 @@ write_pph_file_object (pph_stream *stream, cpp_idents_used *idents_used) { int flags = 0; pth_save_identifiers (idents_used, stream); - fprintf (stream->file, "\n====\n"); - /* FIX pph: Wrong format for writing decls. */ write_pph_namespace (stream, global_namespace, write_pph_print, flags); } @@ -1943,7 +1954,7 @@ write_pph_file (void) if (flag_pph_debug >= 1) fprintf (pph_logfile, "PPH: Writing %s\n", pph_out_file); - stream = pph_stream_open (pph_out_file, false); + stream = pph_stream_open (pph_out_file, "wb"); if (!stream) fatal_error ("Cannot open PPH file for writing: %s: %m", pph_out_file); @@ -2040,8 +2051,8 @@ read_pph_file (const char *filename) if (flag_pph_debug >= 1) fprintf (pph_logfile, "PPH: Reading %s\n", filename); - stream = pph_stream_open (filename, true); - if (!stream->file) + stream = pph_stream_open (filename, "rb"); + if (!stream) fatal_error ("Cannot open PPH file for reading: %s: %m", filename); if (flag_pph_fmt == 0) @@ -2083,7 +2094,7 @@ pth_include_handler (cpp_reader *reader ATTRIBUTE_UNUSED, /* Record a #include or #include_next for PPH. */ static void -pph_include_handler (cpp_reader *reader /*FIXME pph: ATTRIBUTE_UNUSED */, +pph_include_handler (cpp_reader *reader, location_t loc ATTRIBUTE_UNUSED, const unsigned char *dname, const char *name, @@ -2101,7 +2112,7 @@ pph_include_handler (cpp_reader *reader /*FIXME pph: ATTRIBUTE_UNUSED */, } pph_file = query_pph_include_map (name); - if (pph_file != NULL && ! cpp_included_before (reader, name, input_location)) + if (pph_file != NULL && !cpp_included_before (reader, name, input_location)) read_pph_file (pph_file); } -- This patch is available for review at http://codereview.appspot.com/4303049
Sign in to reply to this message.
LTO cleanups. See http://gcc.gnu.org/ml/gcc-patches/2011-03/msg01556.html for details. * lto-streamer-in.c (input_string_internal): Add clarifying comments. * lto-streamer-out.c (lto_output_string_with_length): Rename from output_string_with_length. Output 0 to indicate a non-NULL string. Update all callers to not emit 0. (lto_output_string): Rename from output_string. Make extern. Update all users. diff --git a/gcc/lto-streamer-in.c b/gcc/lto-streamer-in.c index b86008b..d2f0075 100644 --- a/gcc/lto-streamer-in.c +++ b/gcc/lto-streamer-in.c @@ -140,7 +140,10 @@ input_string_internal (struct data_in *data_in, struct lto_input_block *ib, unsigned int loc; const char *result; + /* Read the location of the string from IB. */ loc = lto_input_uleb128 (ib); + + /* Get the string stored at location LOC in DATA_IN->STRINGS. */ LTO_INIT_INPUT_BLOCK (str_tab, data_in->strings, loc, data_in->strings_len); len = lto_input_uleb128 (&str_tab); *rlen = len; diff --git a/gcc/lto-streamer-out.c b/gcc/lto-streamer-out.c index a66e1fb..d3d8294 100644 --- a/gcc/lto-streamer-out.c +++ b/gcc/lto-streamer-out.c @@ -170,6 +170,9 @@ lto_output_string_with_length (struct output_block *ob, s_slot.len = len; s_slot.slot_num = 0; + /* Indicate that this is not a NULL string. */ + lto_output_uleb128_stream (index_stream, 0); + slot = (struct string_slot **) htab_find_slot (ob->string_hash_table, &s_slot, INSERT); if (*slot == NULL) @@ -206,11 +209,8 @@ lto_output_string (struct output_block *ob, const char *string) { if (string) - { - lto_output_uleb128_stream (index_stream, 0); - lto_output_string_with_length (ob, index_stream, string, - strlen (string) + 1); - } + lto_output_string_with_length (ob, index_stream, string, + strlen (string) + 1); else lto_output_uleb128_stream (index_stream, 1); } @@ -225,12 +225,9 @@ output_string_cst (struct output_block *ob, tree string) { if (string) - { - lto_output_uleb128_stream (index_stream, 0); - lto_output_string_with_length (ob, index_stream, - TREE_STRING_POINTER (string), - TREE_STRING_LENGTH (string)); - } + lto_output_string_with_length (ob, index_stream, + TREE_STRING_POINTER (string), + TREE_STRING_LENGTH (string )); else lto_output_uleb128_stream (index_stream, 1); } @@ -245,12 +242,9 @@ output_identifier (struct output_block *ob, tree id) { if (id) - { - lto_output_uleb128_stream (index_stream, 0); - lto_output_string_with_length (ob, index_stream, - IDENTIFIER_POINTER (id), - IDENTIFIER_LENGTH (id)); - } + lto_output_string_with_length (ob, index_stream, + IDENTIFIER_POINTER (id), + IDENTIFIER_LENGTH (id)); else lto_output_uleb128_stream (index_stream, 1); } -- This patch is available for review at http://codereview.appspot.com/4303049
Sign in to reply to this message.
Ugly LTO hacks. To be nuked. See http://gcc.gnu.org/ml/gcc-patches/2011-03/msg01556.html for details. Diego. * lto-streamer-in.c (lto_input_ts_decl_non_common_tree_pointers): Always read DECL_SAVED_TREE. * lto-streamer-out.c (lto_output_ts_decl_common_tree_pointers): Igonore varpool nodes. (lto_output_ts_decl_non_common_tree_pointers): Output NULL DECL_SAVED_TREE. diff --git a/gcc/lto-streamer-in.c b/gcc/lto-streamer-in.c index b86008b..d2f0075 100644 --- a/gcc/lto-streamer-in.c +++ b/gcc/lto-streamer-in.c @@ -2025,6 +2028,10 @@ lto_input_ts_decl_non_common_tree_pointers (struct lto_input_block *ib, { if (TREE_CODE (expr) == FUNCTION_DECL) { + /* FIXME pph - Hookize. */ +#if 1 + DECL_SAVED_TREE (expr) = lto_input_tree (ib, data_in); +#endif DECL_ARGUMENTS (expr) = lto_input_tree (ib, data_in); DECL_RESULT (expr) = lto_input_tree (ib, data_in); } diff --git a/gcc/lto-streamer-out.c b/gcc/lto-streamer-out.c index a66e1fb..d3d8294 100644 --- a/gcc/lto-streamer-out.c +++ b/gcc/lto-streamer-out.c @@ -854,6 +848,8 @@ lto_output_ts_decl_common_tree_pointers (struct output_block *ob, tree expr, && TREE_CODE (expr) != TRANSLATION_UNIT_DECL) { tree initial = DECL_INITIAL (expr); + /* FIXME pph - Hookize. */ +#if 0 if (TREE_CODE (expr) == VAR_DECL && (TREE_STATIC (expr) || DECL_EXTERNAL (expr)) && initial) @@ -866,6 +862,7 @@ lto_output_ts_decl_common_tree_pointers (struct output_block *ob, tree expr, vnode)) initial = NULL; } +#endif lto_output_tree_or_ref (ob, initial, ref_p); } @@ -898,11 +895,16 @@ lto_output_ts_decl_non_common_tree_pointers (struct output_block *ob, { if (TREE_CODE (expr) == FUNCTION_DECL) { +#if 0 /* DECL_SAVED_TREE holds the GENERIC representation for DECL. At this point, it should not exist. Either because it was converted to gimple or because DECL didn't have a GENERIC representation in this TU. */ gcc_assert (DECL_SAVED_TREE (expr) == NULL_TREE); +#else + /* FIXME pph - Hookize and handle FE ASTs. */ + lto_output_tree_or_ref (ob, NULL, ref_p); +#endif lto_output_tree_or_ref (ob, DECL_ARGUMENTS (expr), ref_p); lto_output_tree_or_ref (ob, DECL_RESULT (expr), ref_p); } -- This patch is available for review at http://codereview.appspot.com/4303049
Sign in to reply to this message.
|
