CN112632959B - A method for parsing EPUB files - Google Patents

Abstract

The application relates to an EPUB file analysis method, which comprises the following steps: parsing the meta information: calling a META-information analysis module to analyze a container.xml file in the META-INF file, and reading a root file path of the electronic book; positioning an opf file according to the root file path, and analyzing meta information in the opf file; analyzing the content of the book main body: the book main body content is contained in an xhtml/html file, and the xhtml/html file is classified according to a page processing module and is divided into a cover, a preamble part, a chapter part and a postamble part; then respectively calling a cover analysis unit, a preamble analysis unit and a chapter analysis unit for processing, and respectively semantically processing the three parts of contents; for the postmark part, no processing is performed; extracting the main content of each chapter: the main body content consists of a plurality of sections, each section is wrapped by a section label, and a section label analyzing unit is called for processing. The method can analyze EPUB data into Json format data.

Description

A method for parsing EPUB files

technical field

The invention relates to the technical field of file reading methods, in particular to the semanticization of the EPUB format, specifically to reconstructing books in the EPUB format and converting them into JSON files.

Background technique

EPUB (Electronic publication) format file is a free and open standard, which belongs to a kind of content that can be "automatically rearranged"; that is, the text content can be displayed in the most suitable way for reading according to the characteristics of the reading device. EPUB files internally use XHTML or DTBook (an XML standard proposed by the DAISY Consortium) to display text, and package the file content in zip compression format. The EPUB format includes optional functions related to digital rights management (DRM). However, there is a large amount of redundancy in EPUB e-book data at present, which is not easy to extract and reuse. However, if the book information can be converted into a Json format with strong scalability and simple parsing while retaining the book information, it will be easily converted into other types of files, which has broad application prospects for the conversion and display of e-book information.

The existing EPUB parsing solution is to parse EPUB format files based on epublib-core-latest.jar. Generate file header converter, page converter and object converter according to the parsed information, where the file header converter is used to obtain the author and book name of the file, the page converter is used to obtain the content of the file, and the object converter is used to convert the file header converter , Integration of page conversion information.

The disadvantage of the above prior art is that the data obtained after parsing through epublib is rich text content in html format. If plain text is required, the text content of the label can be obtained through jsoup, but the typesetting of the plain text after acquisition will be messy. Moreover, the steps of generating file header converters, page converters and object converters according to the parsed information are too cumbersome, and the scalability is not strong.

Contents of the invention

The purpose of the present invention is: in order to solve the above-mentioned problem, the present invention proposes a kind of EPUB file parsing method, the purpose is to solve the problem that EPUB e-book data has a large amount of redundancy and is not easy to extract and utilize, and EPUB data is parsed into strong expansibility and small volume. , which is rich in expressiveness and usually easy to convert into semantic Json format data in pdf, word and other formats.

In order to solve the above problems, the technical solution adopted in the present invention is:

A kind of EPUB file parsing method, call decompression module to change the name of EPUB file into zip compressed file and decompress it, then analyze for decompressed file, it is characterized in that, described parsing method comprises the steps:

(1) Analyzing meta information: call the meta information parsing module to parse the container.xml file in the META-INF file, read the root file path of the e-book; locate the opf file according to the root file path, and parse the meta information in the opf file;

(2) Analyze the main content of the book: the main content of the book is included in the xhtml/html file, and the xhtml/html file is classified according to the page processing module. The classification method adopts the epub-type attribute and the data-type attribute of the root tag of the file, According to the different values, it is divided into cover, preface, chapter and postscript; then call the cover analysis unit, preface analysis unit, and chapter analysis unit to process the content of these three parts respectively; for the postscript part, it will not be processed;

(3) Extract the main content of each chapter: the main content is composed of several sections, each section is wrapped by a section tag, and the section tag parsing unit is called for processing.

Further, the analysis of the Opf file is divided into two parts, one is to analyze the meta information of the book, and the other is to analyze the path of the book media file.

Further, the meta information of the book is included in the tag named metadata, and part of the tag information that needs to be parsed is as follows:

The media file path of the book is included in the tag named manifest. Each item tag represents a media file. Only the item tag whose attribute media-type is "application/xhtml+xml" is parsed to obtain the path of all xhtml/html files. , the parsing method is to call the meta information parsing module to build the opf file into a document tree, traverse and process the metadata and manifest tags, extract the value of the text node corresponding to the meta information tag and store it in the predefined semantic Json structure, xhtml and html files The path is stored as an instance variable.

Furthermore, the parsing method of the cover parsing unit is to directly locate and extract the cover image in the file, store it in a specified folder and name it with its md5 value, and the specific method is to search for the img tag in the document tree and pass it The src attribute locates the image directory for moving and calculating the md5 value for renaming.

Furthermore, the parsing method of the preamble parsing unit is to locate the tag whose epub-type is frontmatter, and then traverse all the tags under it, convert them into different Json fields according to the class attribute value, and parse the five types of tags, which are the class is BookTitlePage ,CopyrightPage, Dedication, Preface and BookAcknowledgments tags include the book title, copyright, preface, preface and thank you information respectively. The method of parsing is to perform recursive traversal and convert it into pre-set semantic Json according to the structure of the tags.

Further, the chapter parsing unit decomposes the parsed chapter class xhtml into two parts for extraction; the first is the title and commentary of each chapter, the title part is the h1-h4 tag, and the extraction method is to locate and extract the text node value according to the tag name , the comment part is similar to ordinary paragraphs in EPUB, so the paragraph analysis unit is used: p tag analysis unit or div analysis unit for analysis, where p tag represents simple text, div tag represents more complex text; P tag analysis unit and div tag parsing unit is the entrance of other parsing units, and also the core of chapter parsing unit.

Further, for the P tag parsing unit, the specific steps include: first traversing its child nodes, if it is a text node, store it in the list, otherwise call the processing unit of the corresponding tag for processing, and finally construct a data in the form of a key value , the data format is extremely scalable.

Further, for the div tag parsing unit, the specific steps include: first traverse the div tag to take out all sub-tags under it, and transfer its content to a different processing module for processing. The processing here is different from the key in the parsing result of the P tag parsing module It is always "p", but returns it as a separate key-value pair according to its class, which is not possible with the general Html-to-Json method, and also removes redundant div tags.

The beneficial effects of the above-mentioned technical solution provided by the present invention at least include: the present invention can solve the utilization problem of EPUB e-book information, and by converting it into a semantic Json file, while retaining the book information to the greatest extent, the storage cost is reduced to an extremely low level, And because the Json format is highly extensible and easy to parse, it can be easily converted into other types of files.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

The technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, and are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the attached picture:

Fig. 1 is the parsing flowchart of the EPUB file parsing method disclosed by the embodiment of the present invention.

Fig. 2 is a main block diagram of the EPUB file parsing method disclosed by the embodiment of the present invention.

Fig. 3 is a schematic diagram of the xhtml structure of the chapter parsing unit disclosed in the embodiment of the present invention.

Fig. 4 is an analysis flow chart of the P label analysis unit disclosed in the embodiment of the present invention.

Fig. 5 is a flow chart of parsing by the div tag parsing unit disclosed in the embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

The invention relates to an EPUB file parsing method, the parsing steps are shown in Figure 1, and the main modules are shown in Figure 2. First call the decompression module to rename the EPUB file into a zip compressed file and decompress it, and then perform a comprehensive analysis on the decompressed file.

The first step is to parse the meta information. Generally, there will be three types of files after decompression of e-books following the EPUB3.0 standard, META-INF files, OBEPS (OPS) files and mimetype files. The present invention first calls the meta-information analysis module to analyze the container.xml file in the META-INF file, and reads out the path of the root file (rootfile) of the e-book. Then locate the opf file according to the root file path, and parse the meta information in the opf file. Opf file parsing is divided into two parts, one is to parse the meta information of the book, and the other is to parse the path of the book media file. The meta information of a book is mainly contained in the tag named metadata, and part of the tag information that needs to be parsed is as follows:

meta information name tag name Attributes ISBN Identifier book title Title Id=pub-title book subtitle Title Id=pub-subtitle language of the book Language author Creator publishing house Publisher Copyright Information Rights

The media file path of the book is mainly included in the tag named manifest. Each item tag represents a media file. Only the item tag whose attribute media-type is "application/xhtml+xml" is parsed to obtain all xhtml/html files. Path, the main method of parsing is to call the meta information parsing module to build the opf file into a document tree, traverse and process the metadata and manifest tags, extract the value of the text node corresponding to the meta information tag and store it in the predefined semantic Json structure, xhtml and html file paths are stored as instance variables.

The second step is to analyze the main content of the book. Book main content is included in the xhtml/html file, the present invention first classifies the xhtml/html file according to the page processing module, the main method of classification is to obtain the epub-type attribute and the data-type attribute of its file root tag, according to its The value is different to classify. It is divided into cover, preface, chapter and postscript, and then different modules are called for processing. This is the key innovation part of the present invention, and the content of the book is divided into three parts for semanticization.

The analysis method of the cover analysis unit is to directly locate and extract the cover image in the file, store it in the specified folder and name it with its md5 value, and the specific method is to search for the img tag in the document tree and locate it through its src attribute Move to the picture directory and calculate the md5 value to rename.

The main steps of the preface parsing unit are to locate the label whose epub-type is frontmatter, then traverse all the labels under it, and convert them into different Json fields according to the class attribute value, and mainly analyze five types of labels, which are BookTitlePage and CopyrightPage respectively The tags of ,Dedication, Preface and BookAcknowledgments respectively include information such as book title, copyright, preface, foreword, and thanks.

The chapter parsing unit, the structure of the chapter class xhtml parsed by it is shown in Figure 3, and the present invention decomposes it into two parts for extraction. The first is the title and commentary of each chapter. The title part is generally h1-h4 tags. The extraction method is to locate and extract the text node value according to the tag name. The commentary part is similar to a normal paragraph in EPUB, so a paragraph analysis unit is used. For example: The p tag parsing unit or the div parsing unit performs parsing, where the p tag represents simple text, and the div tag represents more complex text. The P tag parsing unit and the div tag parsing unit are almost the entrances of other parsing units, and they are also the core of the chapter parsing unit.

For the P tag parsing unit, the specific steps are shown in Figure 4. First traverse its child nodes, if it is a text node, store it in the list, otherwise call the processing unit of the corresponding label for processing, and finally construct a key-value form of data, for example: "p":["text",{ "span":["text"]}, {"code":"text"}], this data format is extremely expandable. Note that although the format of the processing results of different tags is almost the same, the processing methods of different tags not exactly.

For the Div tag parsing unit, the specific sub-steps are shown in Figure 5. First, traverse the div tag to take out all the sub-tags under it, and transfer its content to different processing modules for processing. The processing here is different from the P tag parsing module. The key in the parsing result is always "p", but according to its class. Return as a single key-value pair, for example: <div><div class="OrderedList"><ul><li>text value</li></li></div></div> extraction result It is "ol":[{"li":[]},{"li":[]}], which cannot be done by the general Html to Json method, and also removes redundant div tags.

Then extract the main content of each chapter. The main content is composed of several sections, and each section is wrapped by a section tag. The structure under it is similar to the chapter structure, so it will recursively call the section tag analysis unit for processing, section The tag parsing unit is similar to the chapter parsing unit, the difference is that this unit not only recognizes div and p tags to call corresponding units for processing, but also recognizes and parses tags such as figure, ol, ul, and pre for processing.

For the postscript part, considering that its information value is not high, the present invention does not process it.

The present invention also synchronously develops a semantic Json-to-html module, which can convert Json data into an html file. Compared with the html file of the source EPUB file, there is no information loss and is more concise, which reflects the simplicity of the semantic Json format and is easy to convert for other file formats.

It is understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy described.

In the foregoing Detailed Description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, the invention lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby expressly incorporated into the Detailed Description, with each claim standing on its own as a separate preferred embodiment of this invention.

Those skilled in the art should also understand that various illustrative logical blocks, modules, circuits and algorithm steps described in conjunction with the embodiments herein can be implemented as electronic hardware, computer software or a combination thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The steps of the method or algorithm described in conjunction with the embodiments herein may be directly embodied as hardware, a software module executed by a processor, or a combination thereof. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be a component of the processor. The processor and storage medium can be located in the ASIC. The ASIC may be located in the user terminal. Of course, the processor and the storage medium may also exist in the user terminal as discrete components.

For a software implementation, the techniques described in this application can be implemented with modules (eg, procedures, functions, and so on) that perform the functions described herein. These software codes can be stored in memory units and executed by processors. The memory unit can be implemented within the processor or external to the processor, in which case it can be communicatively coupled to the processor via various means as is known in the art.

The foregoing description includes illustrations of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods to describe the above-mentioned embodiments, but those skilled in the art should recognize that various embodiments can be further combined and permuted. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "comprises" is used in the specification or claims, the word is encompassed in a manner similar to the term "comprises" as interpreted when "comprises" is used as a link in the claims. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".

Claims (5)

1. a kind of EPUB file parsing method, call decompression module to change the name of EPUB file into zip compressed file and decompress it, then analyze for decompressed file, it is characterized in that, described parsing method comprises the steps: (1) Analyzing meta information: call the meta information parsing module to parse the container.xml file in the META-INF file, read the root file path of the e-book; locate the opf file according to the root file path, and parse the meta information in the opf file; (2) Analyze the main content of the book: the main content of the book is included in the xhtml/html file, and the xhtml/html file is classified according to the page processing module. The classification method adopts the epub-type attribute and the data-type attribute of the root tag of the file, According to the different values, it is divided into cover, preface, chapter and postscript; then call the cover analysis unit, preface analysis unit, and chapter analysis unit to process the content of these three parts respectively; for the postscript part, it will not be processed; The analysis method of the cover analysis unit is to directly locate and extract the cover image in the file, store it in the specified folder and name it with its md5 value, and the specific method is to search for the img tag in the document tree and locate it through its src attribute Go to the picture directory to move and calculate the md5 value to rename; The parsing method of the preface parsing unit is to locate the tag whose epub-type is frontmatter, then traverse all the tags under it, convert them into different Json fields according to the class attribute value, and parse the five types of tags, respectively, the class is BookTitlePage, CopyrightPage, The tags of Dedication, Preface, and BookAcknowledgments include the book title, copyright, preface, preface, and thank you information respectively. The method of parsing is to perform recursive traversal, and convert it into a pre-set semantic Json according to the structure of the tag; Chapter parsing unit, the parsed chapter class xhtml is decomposed into two parts for extraction; the first is the title and commentary of each chapter, the title part is h1-h4 tags, the extraction method is to locate and extract the text node value according to the tag name, and the commentary part In EPUB, it is similar to ordinary paragraphs, so the paragraph parsing unit: p tag parsing unit or div parsing unit is used for parsing, where p tag represents simple text, div tag represents more complex text; P tag parsing unit and div tag The analysis unit is the entrance of other analysis units and the core of the chapter analysis unit; (3) Extract the main content of each chapter: the main content is composed of several sections, each section is wrapped by a section tag, and the section tag parsing unit is called for processing. 2. a kind of EPUB file parsing method as claimed in claim 1 is characterized in that, Opf file parsing is divided into two parts, and the one, parsing book meta-information, the 2nd, parsing book media file path. 3. A kind of EPUB file parsing method as claimed in claim 2, it is characterized in that, the meta-information of book is included among the label named metadata, wherein the partial label information that needs parsing is as follows: meta information name tag name Attributes ISBN Identifier book title Title Id=pub-title book subtitle Title Id=pub-subtitle language of the book Language author Creator publishing house Publisher Copyright Information Rights The media file path of the book is included in the tag named manifest. Each item tag represents a media file. Only the item tag whose attribute media-type is "application/xhtml+xml" is parsed to obtain the path of all xhtml/html files. , the parsing method is to call the meta information parsing module to build the opf file into a document tree, traverse and process the metadata and manifest tags, extract the value of the text node corresponding to the meta information tag and store it in the predefined semantic Json structure, xhtml and html files The path is stored as an instance variable. 4. a kind of EPUB file parsing method as claimed in claim 1, it is characterized in that, for the P label parsing unit, concrete steps comprise: at first its child node is traversed, if then be stored in the list if it is a text node, otherwise call The processing unit corresponding to the tag performs processing, and finally constructs data in the form of a key value. This data format is extremely expandable. 5. a kind of EPUB file parsing method as claimed in claim 1 is characterized in that, for the div tag parsing unit, concrete steps comprise: at first traversing the div tag takes out all sub-tags under it, and its content is handed over to different processing modules For processing, the processing here is different from the P tag parsing module, where the key in the parsing result is always "p", but returns it as a separate key-value pair according to its class, which cannot be done by the general Html to Json method Arrived, and also removed redundant div tags.