US20190259104A1 - Computer-implemented methods, computer-readable media, and systems for identifying causes of loss - Google Patents
Computer-implemented methods, computer-readable media, and systems for identifying causes of loss Download PDFInfo
- Publication number
- US20190259104A1 US20190259104A1 US16/229,261 US201816229261A US2019259104A1 US 20190259104 A1 US20190259104 A1 US 20190259104A1 US 201816229261 A US201816229261 A US 201816229261A US 2019259104 A1 US2019259104 A1 US 2019259104A1
- Authority
- US
- United States
- Prior art keywords
- insurance
- computer
- loss
- data
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/08—Insurance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3347—Query execution using vector based model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/18—Legal services
Definitions
- the present invention relates generally to computer-implemented methods, computer-readable media, and systems for identifying causes of loss from unstructured or semi-structured insurance claims data.
- Insurers and re-insurers seek a better understanding of accidents in order to lower the incidence of accidents.
- most claims include a text description generated by a human (e.g., a claims representative, a policyholder, a police officer, and the like). These descriptions pose an obstacle to understanding the underlying causes of accidents across the large (e.g., thousands of claims per year) data sets generated by insurance companies.
- One aspect of the invention provides a computer-implemented method of identifying causes of loss from insurance claims data including a plurality of unstructured or semi-structured insurance claims.
- the computer-implemented method includes: loading insurance claims data including a plurality of unstructured or semi-structured insurance claims into memory on a computer; for each of at least a subset of the insurance claims within the insurance claims data, creating a corresponding pre-processed claim record by: tokenizing the insurance claims loaded into memory to separate words in the insurance claims loaded into memory from punctuation, lemmatizing the words in the insurance claims loaded into memory to map morphological variations onto a common base word, removing stop words from the insurance claims loaded into memory, removing punctuation and numbers from the insurance claims loaded into memory, and replacing abbreviations and common typographical errors with associated words previously-defined in a data dictionary stored in memory; creating a Term Frequency-Inverse Document Frequency (TF-IDF) matrix in memory detailing relative frequency of a plurality of n-word terms within at
- n can be an integer between 1 and 6.
- the selecting step can include applying a chi-squared test for each of the plurality of features within the TF-IDF matrix.
- the computer-implemented method can further include applying the binary classifiers against a plurality of the pre-processed claim records not previously used in creating the binary classifiers to identify one or more causes of loss.
- the computer-implemented method can further include, if one or more of the insurance claims was not classified by any of the binary classifiers, designating the insurance claim as unclassifiable.
- the computer-implemented method can further include selecting a single cause of loss from the one or more causes of loss identified by the binary classifiers based upon a pre-defined hierarchy.
- the binary classifiers can be applied in a previously specified priority order.
- the binary classifiers can be applied to identify a single cause of loss.
- the computer-implemented method can further include identifying whether a plurality of the insurance claims was preventable, non-preventable, or partially preventable based on previously stored associations between causes of loss and prevention techniques.
- the computer-implemented method can further include identifying one or more prevention techniques for a plurality of the insurance claims identified as preventable or partially preventable based on previously stored associations between causes of loss and prevention techniques.
- the computer-implemented method can further include identifying which of the one or more prevention techniques are associated with a highest aggregate or average loss over the plurality of preventable or partially preventable insurance claims.
- the computer-implemented method can further include discarding terms having a total frequency over the plurality of insurance claims of less than or equal to 2 before either creating the TF-IDF matrix or selecting a plurality of features from the TF-IDF matrix.
- Another aspect of the invention provides a computer-implemented method of identifying causes of loss from insurance claims data comprising a plurality of unstructured or semi-structured insurance claims.
- the computer-implemented method includes: loading insurance claims data comprising a plurality of unstructured or semi-structured insurance claims into memory on a computer; for each of at least a subset of the insurance claims within the insurance claims data, creating a corresponding pre-processed claim record by: tokenizing the insurance claims loaded into memory to separate words in the insurance claims loaded into memory from punctuation, lemmatizing the words in the insurance claims loaded into memory to map morphological variations onto a common base word, removing stop words from the insurance claims loaded into memory, removing punctuation and numbers from the insurance claims loaded into memory, and replacing abbreviations and common typographical errors with associated words previously-defined in a data dictionary stored in memory; applying the binary classifiers created using the methods described herein against a plurality of the pre-processed claim records to identify one or more causes of loss for each of
- the computer-implemented method can further include, if one or more of the insurance claims was not classified by any of the binary classifiers, designating the insurance claim as unclassifiable.
- the computer-implemented method can further include selecting a single cause of loss from the one or more causes of loss identified by the binary classifiers based upon a pre-defined hierarchy.
- Another aspect of the invention provides a system for identifying causes of loss from insurance claims data comprising a plurality of unstructured or semi-structured insurance claims data.
- the system includes: a processor; and computer-readable memory containing instructions to: implement an interface programmed to receive insurance claims data comprising one or more insurance claims; store the insurance claims data in the computer-readable memory; and invoke execution of the method as described herein on the processor.
- FIG. 1A depicts a method for building a plurality of binary classifiers to identify causes of loss for textual data according to an embodiment of the invention.
- FIG. 1B depicts a method for identifying causes of loss for textual data according to an embodiment of the invention.
- FIG. 2 depicts a computer system through which provisioned textual data can be interfaced, stored, and processed to identify causes of loss and/or mitigation techniques according to an embodiment of the invention.
- FIG. 3 depicts a system for identifying causes of loss and/or accompanying mitigation techniques for textual data according to an embodiment of the invention.
- FIG. 4 depicts a method of generating a Term Frequency-Inverse Document Frequency (TF-IDF) matrix according to an embodiment of the invention.
- TF-IDF Term Frequency-Inverse Document Frequency
- insurance claims data represents the data for which a loss analysis is requested regardless of whether or not it was received from insurance companies, their system processors, their agents, insurance clients and/or their brokers, individual fleets or loss mitigation product vendors.
- Ranges provided herein are understood to be shorthand for all of the values within the range.
- a range of 1 to 50 is understood to include any number, combination of numbers, or sub-range from the group consisting 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, or 50 (as well as fractions thereof unless the context clearly dictates otherwise).
- unstructured data includes information that either does not have a pre-defined data model or whose textual components are not organized in a pre-defined manner.
- structured data includes unstructured data within a data model or other pre-defined manner.
- insurance claims data may be in a data model in which certain fields (e.g., date of accident, driver, automobile make and/or model, rental status) are well-defined, but other data (e.g., accident description) accept free-form text.
- Embodiments of the invention provide computer-implemented methods, computer-readable media, and systems for identifying causes of loss from textual data. Although described in the context of automotive insurance claims, it could be applied to other insurance lines, e.g., casualty insurance, health insurance, disability insurance, life insurance, property insurance, aviation insurance, boiler insurance, builder's risk insurance, crop insurance, home insurance, landlord insurance, marine insurance, liability insurance, and the like.
- Embodiments of the invention are particularly useful for re-insurers that may seek to better understand their underwriting risks and associated loss mitigation potential and/or share this data with their insurance company clients.
- each individual insurance company client used internally consistent styles for describing accidents—a highly unlikely condition—it is also highly unlikely that such styles would be identical across insurance companies or the agencies they contract with to process their losses.
- inconsistencies can arise from at least three sources.
- different people will often use different language for a similar story.
- Such variations can exist on an individual, generational, and/or regional level.
- the report authors may have access to differing amounts of information and may perceive different information to be relevant or irrelevant.
- some companies have their unique loss codes or cause-of-loss buckets.
- Product manufacturers or individual fleets e.g., rental car fleets, government vehicles, and the like
- the incident/accident data can be unstructured or semi-structured.
- the insurance claims data can include human-generated descriptions of claims, examples of which are provided in Table 1 below.
- embodiments of the invention are particularly useful in processing unstructured data, structured data associated with an insurance claim can also be utilized.
- structured data fields detailing the time of accident, age of driver, vehicle type, vehicle usage, and the like can also be utilized to identify features for use in building binary classifiers.
- one embodiment of the invention provides a computer-implemented method for identifying causes of loss from insurance data.
- the method described in FIGS. 1A and 1B can be practiced in a single instance, it need not and, in many instances, will not be practiced in a single instance. Instead, the method of FIG. 1A will often be performed once using a training data set in which a cause of loss was previously specified (e.g., by humans) to build classifiers before the method of FIG. 1B is applied to client-provisioned data.
- 1A can be repeated for each data set (e.g., each time an insurance client provides data for analysis), periodically (e.g., every week, month, year, and the like), upon introduction of new causes of loss (e.g., for accidents deemed unclassifiable), and the like.
- step S 102 insurance claims data is loaded into memory.
- the insurance data can be transferred from various media including storage disks using various technologies.
- the insurance claims data can be submitted over the internet in a variety of file types.
- step S 104 the insurance claims can be preprocessed before it is mapped onto a meaningful computation representation, such as a vectorial representation, used in later steps of the methods.
- Preprocessing maps claims onto a list of tokens that have linguistic meaning, e.g., n-word terms, that can then be classified in further steps.
- Preprocessing can include one or more of the following steps: tokenization (S 104 a ), lemmatization (S 104 b ), stop word removal (S 104 c ), punctuation and number removal (S 104 d ), and replacement of abbreviations and common typographical errors (colloquially, “typos”) (S 104 e ).
- Tokenization (S 104 a ) separates punctuation from words.
- lemmatization maps each morphological variation of a word to its base form. For example, the words, “go”, “going”, “went”, and “gone” are lemmatized to their root or base form “go”.
- Lemmatizers include the WORDNET® system, available from Princeton University of Princeton, N.J. Other lemmatizers can be used, including the MORPHATM software described in G. Minnen et al., Applied morphological processing of English, 7(3) Natural Language Engineering 207-23 (2001) and available at http://www.informatics.susx.ac.uk/research/groups/nlp/carroll/morph.html.
- stop words are common words that appear in too many documents and, therefore, do not have discriminative power. That is, stop words cannot be used to capture the essence of a document such that one can differentiate one document from another.
- Standard lists of stop words are provided in software programs such as the SMART Information Retrieval System, available from Cornell University of Ithaca, N.Y. The SMART stop word list is available at ftp://ftp.cs.cornell.edu/pub/smart/english.stop.
- a collection of stop words for a particular set of documents can be manually selected using domain knowledge or created by selecting the words that occur in more than 80% of the documents. See R. Baeza-Yates & B. Ribeiro-Neto, Modern Information Retrieval ⁇ 7.2 (1999).
- PR an abbreviation for “police report” may be a stop word in an automotive accident data set.
- Removal of numbers in step S 104 d can reduce overfitting in later steps caused by inherent ambiguity in numbers.
- the number “75” could represent a vehicle speed, a compass bearing, a street address number, a route number, a vehicle make year, and the like.
- Step S 104 e can use a custom dictionary reflecting domain knowledge (e.g., of the automotive insurance industry, of a particular insurance client, and the like). For example, in “id strikk cv”, “id” should be “IV” (“insured vehicle”) and “strikk” should be “struck”, indicating that an insured vehicle struck claimant vehicle. Additionally or alternatively, the model can adjust for regional phraseology such as determining that “driver ran the light” has the same meaning the same as “driver did not stop for light”.
- step S 104 a variety of free-form text descriptions may converge to a common format.
- a description of “IV Rearend-ed CV” (along with variations thereof) can be cleaned to “Insur vehicl rear end claimant vehicl” (or other descriptions based on the custom dictionary).
- TF-IDF Term Frequency-Inverse Document Frequency
- Insurance claims can be mapped onto a vectorial representation where each report is a
- the vocabulary can be obtained by extracting unique terms after preprocessing the collection.
- dimensions corresponds to a unique term in the collection and the value along the dimension is a weight that indicates the importance of the term/dimension for a particular report d j .
- d j [ w 1 ,w 2 ,w 3 , . . . ,w
- w i is the weight of term i in document d j .
- w i is the weight of term i in document d j .
- the corresponding value in the vector for the term is zero.
- TF-IDF is a composed measure that is the product of the frequency of a term in a document (TF) and its inversed document frequency (IDF).
- TF-IDF provides a metric measuring the importance of term. A term will receive a high value if it frequently appears in the document (TF) and does not occur often in other documents (IDF).
- n i,j is the number of occurrences of the considered term in document d j and the denominator is the frequency of the most frequent term in document d j .
- the denominator is used to normalize the TF values. Other normalization factors can be used such as the most-frequent term in any document in the collection or the number of occurrences of all terms in document d j .
- the IDF of a term is the percentage of distinct documents the term appears in amongst a collection of documents.
- the IDF measures the specificity of a term. The fewer documents a term appears in, the more specific the term is.
- the IDF can be expressed mathematically as:
- IDF i log ⁇ ⁇ D ⁇ ⁇ ⁇ d j ⁇ : ⁇ t i ⁇ d j ⁇ ⁇ ( 2 )
- N represents the total number of reports in the collection
- df(t) represents the number of reports containing the term t.
- FIG. 4 An exemplary implementation of a TF-IDF matrix is depicted in FIG. 4 .
- a plurality of features are selected from the TF-IDF matrix. For example, a chi-squared test can be used to identify a difference in expected frequency of a cause of loss in the training set relative to the observed frequency of the cause of loss when a feature is present. The m n-grams having the largest chi-squared value can be selected for further use.
- step S 110 binary classifiers are created for each of a plurality of causes of loss.
- a binary classifier divides each of a set of insurance claims into two groups: YES or NO for each potential cause of loss.
- binary classifier algorithms could be used instead of binary classifier algorithms. However, binary classifier algorithms are preferred because accident data tends to be unbalanced across cause of loss. For example, 40% of all accidents may be rear-end collisions and 30% may involve a parked car, while other causes of loss may occur in less than 10% of insurance claims. Multiple classifiers would prefer these larger classification “buckets” and tend to classify an insurance claim as the most prevalent cause of loss, even though other rarer causes of loss may also be present. Binary classification identifies all causes of loss and allows for optional later culling to one or more causes of loss in step S 120 .
- the binary classifiers built in the method of FIG. 1A can now be used to classify unseen insurance claims data.
- the initial loading and pre-processing steps S 112 and S 114 (including steps S 114 a -S 114 e ) can be similar or identical to steps S 102 and S 104 .
- loading and pre-processing can be performed once.
- insurance claims data can be received from an insurance client, loaded, pre-processed, and divided into training and real-world data sets in any order.
- step S 116 each binary classifier built in step S 110 can be applied against each insurance claim. This may produce a plurality of positive answers for a single insurance claim as seen in Table 2.
- a priority or hierarchy of causes of loss can be applied to identify and associate a single cause of loss with each insurance claim.
- an IF-ELSE loop can be applied over each insurance claim in the order of priority to check if the claim was classified as cause of loss 1, 2, . . . , n.
- that cause of loss can be associated with the insurance claim and the loop can terminate.
- searches can be performed iteratively for each cause of loss in order of priority.
- the priority order is set based on the number of classifications made in step S 118 .
- the priority order can be in order from most-to-least or least-to-most inclusive classifiers.
- the binary classifiers can be applied in step S 116 in order of priority (e.g., the highest priority classifier first), with further binary classifiers only applied to the sub-set of insurance claims not yet classified by a classifier for a higher priority cause of loss.
- Some insurance claims may not yield a positive answer for any cause of loss and can be designated as “unclassifiable” in step S 120 .
- String matching and/or human review can be performed to identify one or more causes of loss and such data can be used for further training of the binary classifiers in S 110 .
- the classifications produced by the invention are valuable in and of themselves.
- the classifications can also be augmented to provide actionable advice to insurance and re-insurance companies.
- the distribution of causes of loss produced by the methods described herein can be married with cost data from the insurance claims (e.g., by division) to identify which causes of loss are associated with the highest average cost per claim.
- costs associated with the insurance claims or industry averages can be associated with the classifications (e.g., by multiplication) to identify which causes of loss result in the largest aggregate costs over a portfolio.
- the cause of loss distribution can be married with data regarding prevention techniques (e.g., forward collision warning device, driver training, rear backup camera, lane departure warning device, blind spot monitoring device, motorcycle warning device) and associated costs and risk reduction to identify the largest opportunities for cost savings. Additionally or alternatively, costs or risk exposure can be presented graphically (e.g., in bar or pie charts) or on numerical scale (e.g., exposure due to each cause of loss on a 0-10 scale).
- prevention techniques e.g., forward collision warning device, driver training, rear backup camera, lane departure warning device, blind spot monitoring device, motorcycle warning device
- costs or risk exposure can be presented graphically (e.g., in bar or pie charts) or on numerical scale (e.g., exposure due to each cause of loss on a 0-10 scale).
- FIG. 2 another embodiment of the invention provides a system 200 for implementing the methods described herein.
- An interface 204 can support communication with a data source 202 , which can be an internal or an external client.
- interface 204 can be a Web page or other internet technology that allow customers to upload insurance claims data.
- the interface 204 can verify compliance with various input requirements.
- the interface 204 can also relay the information to storage/memory 206 and/or processor 208 .
- the interface 204 can invoke one or more processes for execution by the processor 208 .
- Processor 208 can load and execute one or more functions, methods, modules, objects or other computing structures to implement the methods described herein. For example, data pre-processing, TF-IDF generation, feature selection, binary classifier generation, binary classifier application, and single cause of loss identification can be each implemented by a separate function/method/module 210 a , 210 b , 210 c , 210 d , 210 e , and 210 f , respectively.
- Such separate functions/methods/modules 210 a , 210 b , 210 c , 210 d , 210 e , 210 f can be invoked by an overarching function/method/module that can pass the result of a previous function/method/module to the next function/method/module, e.g., by reference to the resulting data.
- embodiments of the invention utilizing a plurality of binary classifiers are more accurate than models utilizing a single multiple classifier as discussed in greater detail herein.
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- Finance (AREA)
- Economics (AREA)
- Technology Law (AREA)
- Accounting & Taxation (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Primary Health Care (AREA)
- Human Resources & Organizations (AREA)
- Development Economics (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
Abstract
Description
- This application claims the benefit of priority to U.S. Provisional Patent Application Ser. No. 62/631,579, filed Feb. 16, 2018. The entire content of this application is hereby incorporated by reference herein.
- The present invention relates generally to computer-implemented methods, computer-readable media, and systems for identifying causes of loss from unstructured or semi-structured insurance claims data.
- Insurers and re-insurers seek a better understanding of accidents in order to lower the incidence of accidents. At least in the auto insurance field, most claims include a text description generated by a human (e.g., a claims representative, a policyholder, a police officer, and the like). These descriptions pose an obstacle to understanding the underlying causes of accidents across the large (e.g., thousands of claims per year) data sets generated by insurance companies.
- One aspect of the invention provides a computer-implemented method of identifying causes of loss from insurance claims data including a plurality of unstructured or semi-structured insurance claims. The computer-implemented method includes: loading insurance claims data including a plurality of unstructured or semi-structured insurance claims into memory on a computer; for each of at least a subset of the insurance claims within the insurance claims data, creating a corresponding pre-processed claim record by: tokenizing the insurance claims loaded into memory to separate words in the insurance claims loaded into memory from punctuation, lemmatizing the words in the insurance claims loaded into memory to map morphological variations onto a common base word, removing stop words from the insurance claims loaded into memory, removing punctuation and numbers from the insurance claims loaded into memory, and replacing abbreviations and common typographical errors with associated words previously-defined in a data dictionary stored in memory; creating a Term Frequency-Inverse Document Frequency (TF-IDF) matrix in memory detailing relative frequency of a plurality of n-word terms within at least a subject of the pre-processed claim record, wherein n is a positive integer; selecting a plurality of features from the TF-IDF matrix; and creating a binary classifier for each of a plurality of causes of loss.
- This aspect of the invention can include a variety of embodiments. For example, n can be an integer between 1 and 6. The selecting step can include applying a chi-squared test for each of the plurality of features within the TF-IDF matrix.
- The computer-implemented method can further include applying the binary classifiers against a plurality of the pre-processed claim records not previously used in creating the binary classifiers to identify one or more causes of loss. The computer-implemented method can further include, if one or more of the insurance claims was not classified by any of the binary classifiers, designating the insurance claim as unclassifiable. The computer-implemented method can further include selecting a single cause of loss from the one or more causes of loss identified by the binary classifiers based upon a pre-defined hierarchy. The binary classifiers can be applied in a previously specified priority order. The binary classifiers can be applied to identify a single cause of loss.
- The computer-implemented method can further include identifying whether a plurality of the insurance claims was preventable, non-preventable, or partially preventable based on previously stored associations between causes of loss and prevention techniques. The computer-implemented method can further include identifying one or more prevention techniques for a plurality of the insurance claims identified as preventable or partially preventable based on previously stored associations between causes of loss and prevention techniques. The computer-implemented method can further include identifying which of the one or more prevention techniques are associated with a highest aggregate or average loss over the plurality of preventable or partially preventable insurance claims.
- The computer-implemented method can further include discarding terms having a total frequency over the plurality of insurance claims of less than or equal to 2 before either creating the TF-IDF matrix or selecting a plurality of features from the TF-IDF matrix.
- Another aspect of the invention provides a computer-implemented method of identifying causes of loss from insurance claims data comprising a plurality of unstructured or semi-structured insurance claims. The computer-implemented method includes: loading insurance claims data comprising a plurality of unstructured or semi-structured insurance claims into memory on a computer; for each of at least a subset of the insurance claims within the insurance claims data, creating a corresponding pre-processed claim record by: tokenizing the insurance claims loaded into memory to separate words in the insurance claims loaded into memory from punctuation, lemmatizing the words in the insurance claims loaded into memory to map morphological variations onto a common base word, removing stop words from the insurance claims loaded into memory, removing punctuation and numbers from the insurance claims loaded into memory, and replacing abbreviations and common typographical errors with associated words previously-defined in a data dictionary stored in memory; applying the binary classifiers created using the methods described herein against a plurality of the pre-processed claim records to identify one or more causes of loss for each of the insurance claims within the insurance claims data.
- This aspect of the invention can have a variety of embodiments. The computer-implemented method can further include, if one or more of the insurance claims was not classified by any of the binary classifiers, designating the insurance claim as unclassifiable. The computer-implemented method can further include selecting a single cause of loss from the one or more causes of loss identified by the binary classifiers based upon a pre-defined hierarchy.
- Another aspect of the invention provides a system for identifying causes of loss from insurance claims data comprising a plurality of unstructured or semi-structured insurance claims data. The system includes: a processor; and computer-readable memory containing instructions to: implement an interface programmed to receive insurance claims data comprising one or more insurance claims; store the insurance claims data in the computer-readable memory; and invoke execution of the method as described herein on the processor.
- For a fuller understanding of the nature and desired objects of the present invention, reference is made to the following detailed description taken in conjunction with the accompanying drawing figures wherein like reference characters denote corresponding parts throughout the several views.
-
FIG. 1A depicts a method for building a plurality of binary classifiers to identify causes of loss for textual data according to an embodiment of the invention. -
FIG. 1B depicts a method for identifying causes of loss for textual data according to an embodiment of the invention. -
FIG. 2 depicts a computer system through which provisioned textual data can be interfaced, stored, and processed to identify causes of loss and/or mitigation techniques according to an embodiment of the invention. -
FIG. 3 depicts a system for identifying causes of loss and/or accompanying mitigation techniques for textual data according to an embodiment of the invention. -
FIG. 4 depicts a method of generating a Term Frequency-Inverse Document Frequency (TF-IDF) matrix according to an embodiment of the invention. - The instant invention is most clearly understood with reference to the following definitions.
- As used herein, the singular form “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
- As used in the specification and claims, the terms “comprises,” “comprising,” “containing,” “having,” and the like can have the meaning ascribed to them in U.S. patent law and can mean “includes,” “including,” and the like.
- Unless specifically stated or obvious from context, the term “or,” as used herein, is understood to be inclusive.
- As used herein, “insurance claims data” represents the data for which a loss analysis is requested regardless of whether or not it was received from insurance companies, their system processors, their agents, insurance clients and/or their brokers, individual fleets or loss mitigation product vendors.
- Ranges provided herein are understood to be shorthand for all of the values within the range. For example, a range of 1 to 50 is understood to include any number, combination of numbers, or sub-range from the group consisting 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, or 50 (as well as fractions thereof unless the context clearly dictates otherwise).
- As used herein, the term “unstructured data” includes information that either does not have a pre-defined data model or whose textual components are not organized in a pre-defined manner. As used herein, the term “semi-structured data” includes unstructured data within a data model or other pre-defined manner. For example, insurance claims data may be in a data model in which certain fields (e.g., date of accident, driver, automobile make and/or model, rental status) are well-defined, but other data (e.g., accident description) accept free-form text.
- Embodiments of the invention provide computer-implemented methods, computer-readable media, and systems for identifying causes of loss from textual data. Although described in the context of automotive insurance claims, it could be applied to other insurance lines, e.g., casualty insurance, health insurance, disability insurance, life insurance, property insurance, aviation insurance, boiler insurance, builder's risk insurance, crop insurance, home insurance, landlord insurance, marine insurance, liability insurance, and the like.
- Although textual data poses processing challenges, it provides a valuable narrative to human readers that process an insurance claim and captures information that could not be efficiently captured in structured fields that must be programmed in advance and become tedious if too numerous.
- Further complicating claims classification across various entities is the nonstandard codification process and data capture methodologies employed by each collecting company. The ability to bypass these differences by analyzing the original sources of the data provides a unique opportunity to gain insight from hundreds of thousands to millions of analyzed records, rather than the just the thousands produced by one company.
- However, if one could discover valuable insight from this textual information, a company could have a better understanding of why the accidents occurred, whether they were preventable, and how. The invention described enables this deeper and automated insight.
- Embodiments of the invention are particularly useful for re-insurers that may seek to better understand their underwriting risks and associated loss mitigation potential and/or share this data with their insurance company clients. In particular, even if each individual insurance company client used internally consistent styles for describing accidents—a highly unlikely condition—it is also highly unlikely that such styles would be identical across insurance companies or the agencies they contract with to process their losses. For example, inconsistencies can arise from at least three sources. First, different people will often use different language for a similar story. Such variations can exist on an individual, generational, and/or regional level. Additionally, the report authors may have access to differing amounts of information and may perceive different information to be relevant or irrelevant. Moreover, some companies have their unique loss codes or cause-of-loss buckets.
- Product manufacturers or individual fleets (e.g., rental car fleets, government vehicles, and the like) can process their loss descriptions outside of the insurance context for similar analysis to determine what type of loss mitigation method might be the most economically feasible for their unique loss situation. While the accident/incident description provided may not be in the context of insurance, the invention's analysis process would be applicable.
- The incident/accident data can be unstructured or semi-structured. For example, the insurance claims data can include human-generated descriptions of claims, examples of which are provided in Table 1 below. Although embodiments of the invention are particularly useful in processing unstructured data, structured data associated with an insurance claim can also be utilized. For example, structured data fields detailing the time of accident, age of driver, vehicle type, vehicle usage, and the like can also be utilized to identify features for use in building binary classifiers.
-
TABLE 1 Example Loss Descriptions INSURED JUST DISCOVERED THAT THE CATALYTIC CONVERTER HAS BEEN STOLEN CATALYTIC CONVERTER HAS BEEN STOLEN OFF THE VEHICLE INSURED ACTIVATED OUTRIGGING ON IV .APPARATUS STRUCK& DAMAGED[. . .] IV WAS PARKED UNKNOWN PARTY HIT FRONT END WINDSHIELD CRACKED BY ROCK THAT GOT KICKED UP INSURED VEHICLE SKIDDED WHILE PLOWING TOWN PARK VEHICLE HIT RAIL IV WENT TO RETRIEVE A PERSONAL ITEM FROM THE SUNGLASS HOLDER, [. . .] JOHN SMITH WAS TRAVELLING DOWN SULLIVAN AVENUE IN SOUTH [. . .] IV STRUCK CV IN THE REAR - Referring now to
FIGS. 1A and 1B , one embodiment of the invention provides a computer-implemented method for identifying causes of loss from insurance data. Although the method described inFIGS. 1A and 1B can be practiced in a single instance, it need not and, in many instances, will not be practiced in a single instance. Instead, the method ofFIG. 1A will often be performed once using a training data set in which a cause of loss was previously specified (e.g., by humans) to build classifiers before the method ofFIG. 1B is applied to client-provisioned data. The training method ofFIG. 1A can be repeated for each data set (e.g., each time an insurance client provides data for analysis), periodically (e.g., every week, month, year, and the like), upon introduction of new causes of loss (e.g., for accidents deemed unclassifiable), and the like. - In step S102, insurance claims data is loaded into memory. The insurance data can be transferred from various media including storage disks using various technologies. For example, the insurance claims data can be submitted over the internet in a variety of file types.
- In step S104, the insurance claims can be preprocessed before it is mapped onto a meaningful computation representation, such as a vectorial representation, used in later steps of the methods. Preprocessing maps claims onto a list of tokens that have linguistic meaning, e.g., n-word terms, that can then be classified in further steps. Preprocessing can include one or more of the following steps: tokenization (S104 a), lemmatization (S104 b), stop word removal (S104 c), punctuation and number removal (S104 d), and replacement of abbreviations and common typographical errors (colloquially, “typos”) (S104 e).
- Tokenization (S104 a) separates punctuation from words.
- Referring to step S104 b, lemmatization maps each morphological variation of a word to its base form. For example, the words, “go”, “going”, “went”, and “gone” are lemmatized to their root or base form “go”. Lemmatizers include the WORDNET® system, available from Princeton University of Princeton, N.J. Other lemmatizers can be used, including the MORPHA™ software described in G. Minnen et al., Applied morphological processing of English, 7(3) Natural Language Engineering 207-23 (2001) and available at http://www.informatics.susx.ac.uk/research/groups/nlp/carroll/morph.html.
- Referring to step S104 c, “stop words” (also called “stopwords” or “noise words”) are common words that appear in too many documents and, therefore, do not have discriminative power. That is, stop words cannot be used to capture the essence of a document such that one can differentiate one document from another. Standard lists of stop words are provided in software programs such as the SMART Information Retrieval System, available from Cornell University of Ithaca, N.Y. The SMART stop word list is available at ftp://ftp.cs.cornell.edu/pub/smart/english.stop. A collection of stop words for a particular set of documents can be manually selected using domain knowledge or created by selecting the words that occur in more than 80% of the documents. See R. Baeza-Yates & B. Ribeiro-Neto, Modern Information Retrieval § 7.2 (1999). For example, “PR” (an abbreviation for “police report”) may be a stop word in an automotive accident data set.
- Removal of numbers in step S104 d can reduce overfitting in later steps caused by inherent ambiguity in numbers. (For example, the number “75” could represent a vehicle speed, a compass bearing, a street address number, a route number, a vehicle make year, and the like.)
- Step S104 e can use a custom dictionary reflecting domain knowledge (e.g., of the automotive insurance industry, of a particular insurance client, and the like). For example, in “id strikk cv”, “id” should be “IV” (“insured vehicle”) and “strikk” should be “struck”, indicating that an insured vehicle struck claimant vehicle. Additionally or alternatively, the model can adjust for regional phraseology such as determining that “driver ran the light” has the same meaning the same as “driver did not stop for light”.
- As a result of step S104, a variety of free-form text descriptions may converge to a common format. For example, a description of “IV Rearend-ed CV” (along with variations thereof) can be cleaned to “Insur vehicl rear end claimant vehicl” (or other descriptions based on the custom dictionary).
- In step S106, a Term Frequency-Inverse Document Frequency (TF-IDF) matrix is created in memory detailing relative frequency of a plurality of n-word terms within each insurance claim, wherein n is a positive integer. The consideration of n-word terms captures the potential significance of phrases such as the 2-word “stop light”, which may connote a more specific meaning than the 1-word term “stop”.
- Insurance claims can be mapped onto a vectorial representation where each report is a |V| dimensional vector, where V is the vocabulary of a large collection of defect reports and |V| is the size of the vocabulary. The vocabulary can be obtained by extracting unique terms after preprocessing the collection. Each dimension in this space of |V| dimensions corresponds to a unique term in the collection and the value along the dimension is a weight that indicates the importance of the term/dimension for a particular report dj. Below is an exemplary representation for insurance claim dj:
-
d j=[w 1 ,w 2 ,w 3 , . . . ,w |V|] (1) - where wi is the weight of term i in document dj. Usually, if a term is not present in a document dj, the corresponding value in the vector for the term is zero. Although the TF-IDF approach is described herein, other weighting schemes can be used.
- TF-IDF is a composed measure that is the product of the frequency of a term in a document (TF) and its inversed document frequency (IDF). TF-IDF provides a metric measuring the importance of term. A term will receive a high value if it frequently appears in the document (TF) and does not occur often in other documents (IDF).
- The TF (“term frequency”) value for term ti within document dj can be expressed mathematically as TFi,j=ni,j or can be normalized using the equation
-
- wherein ni,j is the number of occurrences of the considered term in document dj and the denominator is the frequency of the most frequent term in document dj. The denominator is used to normalize the TF values. Other normalization factors can be used such as the most-frequent term in any document in the collection or the number of occurrences of all terms in document dj.
- The IDF of a term is the percentage of distinct documents the term appears in amongst a collection of documents. The IDF measures the specificity of a term. The fewer documents a term appears in, the more specific the term is. The IDF can be expressed mathematically as:
-
- wherein |D| is the total number of documents in the corpus and |{dj: ti∈dj}| is the number of documents where the term ti appears (that is, ni,j≠0). In another embodiment, IDF(t)=1+log
-
- wherein the logarithm function provides non-linear scaling, N represents the total number of reports in the collection, and df(t) represents the number of reports containing the term t.
- An exemplary implementation of a TF-IDF matrix is depicted in
FIG. 4 . - In step S108, a plurality of features are selected from the TF-IDF matrix. For example, a chi-squared test can be used to identify a difference in expected frequency of a cause of loss in the training set relative to the observed frequency of the cause of loss when a feature is present. The m n-grams having the largest chi-squared value can be selected for further use.
- In step S110, binary classifiers are created for each of a plurality of causes of loss. A binary classifier divides each of a set of insurance claims into two groups: YES or NO for each potential cause of loss.
-
TABLE 2 Binary Classifications for “The insured vehicle made a left turn and hit a parked vehicle” Left turn? YES Hit pedestrian? NO Not at fault? NO . . . . . . Hit parked vehicle? YES - Multiple classifier algorithms could be used instead of binary classifier algorithms. However, binary classifier algorithms are preferred because accident data tends to be unbalanced across cause of loss. For example, 40% of all accidents may be rear-end collisions and 30% may involve a parked car, while other causes of loss may occur in less than 10% of insurance claims. Multiple classifiers would prefer these larger classification “buckets” and tend to classify an insurance claim as the most prevalent cause of loss, even though other rarer causes of loss may also be present. Binary classification identifies all causes of loss and allows for optional later culling to one or more causes of loss in step S120.
- Various binary classifier algorithms can be used including the following listed in Table 3 using a 70%-30% training-testing data split.
-
TABLE 3 Performance Data for Various Binary Classifiers Binary Classifier Overall Accuracy Comments Random Forest 86% Stochastic gradient descent 82% Support vector machine 78% Best performance on (non-linear RBF kernel) large buckets Support vector machine 80% Fastest (linear kernel) - Referring again to
FIG. 1B , the binary classifiers built in the method ofFIG. 1A can now be used to classify unseen insurance claims data. The initial loading and pre-processing steps S112 and S114 (including steps S114 a-S114 e) can be similar or identical to steps S102 and S104. In some embodiments, loading and pre-processing can be performed once. For example, insurance claims data can be received from an insurance client, loaded, pre-processed, and divided into training and real-world data sets in any order. - In step S116, each binary classifier built in step S110 can be applied against each insurance claim. This may produce a plurality of positive answers for a single insurance claim as seen in Table 2.
- In step S118, a priority or hierarchy of causes of loss can be applied to identify and associate a single cause of loss with each insurance claim. For example, an IF-ELSE loop can be applied over each insurance claim in the order of priority to check if the claim was classified as cause of
loss 1, 2, . . . , n. When the highest priority cause of loss is identified, that cause of loss can be associated with the insurance claim and the loop can terminate. In other embodiments, searches can be performed iteratively for each cause of loss in order of priority. - In some embodiments, the priority order is set based on the number of classifications made in step S118. For example, the priority order can be in order from most-to-least or least-to-most inclusive classifiers.
- In still other embodiments, the binary classifiers can be applied in step S116 in order of priority (e.g., the highest priority classifier first), with further binary classifiers only applied to the sub-set of insurance claims not yet classified by a classifier for a higher priority cause of loss.
- Some insurance claims may not yield a positive answer for any cause of loss and can be designated as “unclassifiable” in step S120. String matching and/or human review can be performed to identify one or more causes of loss and such data can be used for further training of the binary classifiers in S110.
- The classifications produced by the invention are valuable in and of themselves. The classifications can also be augmented to provide actionable advice to insurance and re-insurance companies. For example, the distribution of causes of loss produced by the methods described herein can be married with cost data from the insurance claims (e.g., by division) to identify which causes of loss are associated with the highest average cost per claim. Likewise, costs associated with the insurance claims or industry averages can be associated with the classifications (e.g., by multiplication) to identify which causes of loss result in the largest aggregate costs over a portfolio. Furthermore, the cause of loss distribution can be married with data regarding prevention techniques (e.g., forward collision warning device, driver training, rear backup camera, lane departure warning device, blind spot monitoring device, motorcycle warning device) and associated costs and risk reduction to identify the largest opportunities for cost savings. Additionally or alternatively, costs or risk exposure can be presented graphically (e.g., in bar or pie charts) or on numerical scale (e.g., exposure due to each cause of loss on a 0-10 scale).
- Referring now to
FIG. 2 , another embodiment of the invention provides a system 200 for implementing the methods described herein. - An
interface 204 can support communication with adata source 202, which can be an internal or an external client. For example,interface 204 can be a Web page or other internet technology that allow customers to upload insurance claims data. Theinterface 204 can verify compliance with various input requirements. Theinterface 204 can also relay the information to storage/memory 206 and/orprocessor 208. For example, theinterface 204 can invoke one or more processes for execution by theprocessor 208. -
Processor 208 can load and execute one or more functions, methods, modules, objects or other computing structures to implement the methods described herein. For example, data pre-processing, TF-IDF generation, feature selection, binary classifier generation, binary classifier application, and single cause of loss identification can be each implemented by a separate function/method/module modules - Prior to the development of the invention described herein, classification of insurance claims data was a tedious task requiring human labor. Applicant's invention not only minimizes or eliminates the need for human labor and improves accuracy, but also provides several orders of magnitude of speed improvements over human classification. For example, Applicant can classify thousands of insurance claims in seconds.
- Additionally, embodiments of the invention utilizing a plurality of binary classifiers are more accurate than models utilizing a single multiple classifier as discussed in greater detail herein.
- Although preferred embodiments of the invention have been described using specific terms, such description is for illustrative purposes only, and it is to be understood that changes and variations may be made without departing from the spirit or scope of the following claims.
- The entire contents of all patents, published patent applications, and other references cited herein are hereby expressly incorporated herein in their entireties by reference.
- Exemplary PYTHON® source code for an implementation of an embodiment of the methods described herein is provided below.
- lossdetect_Model_run
Claims (16)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/229,261 US20190259104A1 (en) | 2018-02-16 | 2018-12-21 | Computer-implemented methods, computer-readable media, and systems for identifying causes of loss |
US18/472,538 US12100056B2 (en) | 2018-02-16 | 2023-09-22 | Computer-implemented methods, computer-readable media, and systems for identifying causes of loss |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862631579P | 2018-02-16 | 2018-02-16 | |
US16/229,261 US20190259104A1 (en) | 2018-02-16 | 2018-12-21 | Computer-implemented methods, computer-readable media, and systems for identifying causes of loss |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/472,538 Continuation US12100056B2 (en) | 2018-02-16 | 2023-09-22 | Computer-implemented methods, computer-readable media, and systems for identifying causes of loss |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190259104A1 true US20190259104A1 (en) | 2019-08-22 |
Family
ID=67617891
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/229,261 Abandoned US20190259104A1 (en) | 2018-02-16 | 2018-12-21 | Computer-implemented methods, computer-readable media, and systems for identifying causes of loss |
US18/472,538 Active US12100056B2 (en) | 2018-02-16 | 2023-09-22 | Computer-implemented methods, computer-readable media, and systems for identifying causes of loss |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/472,538 Active US12100056B2 (en) | 2018-02-16 | 2023-09-22 | Computer-implemented methods, computer-readable media, and systems for identifying causes of loss |
Country Status (3)
Country | Link |
---|---|
US (2) | US20190259104A1 (en) |
EP (1) | EP3752929A4 (en) |
WO (1) | WO2019160608A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113505228A (en) * | 2021-07-22 | 2021-10-15 | 上海弘玑信息技术有限公司 | Multi-dimensional text data classification method, training method and device |
WO2022135915A1 (en) * | 2020-12-24 | 2022-06-30 | British Telecommunications Public Limited Company | Pre-processing for natural language processing |
CN116150341A (en) * | 2023-04-23 | 2023-05-23 | 之江实验室 | Claim event detection method, computer equipment and storage medium |
Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070282824A1 (en) * | 2006-05-31 | 2007-12-06 | Ellingsworth Martin E | Method and system for classifying documents |
US20080195577A1 (en) * | 2007-02-09 | 2008-08-14 | Wei Fan | Automatically and adaptively determining execution plans for queries with parameter markers |
US20080270268A1 (en) * | 2007-04-27 | 2008-10-30 | Caterpillar Inc. | System and method for service loss analysis and reporting |
US20130054603A1 (en) * | 2010-06-25 | 2013-02-28 | U.S. Govt. As Repr. By The Secretary Of The Army | Method and apparatus for classifying known specimens and media using spectral properties and identifying unknown specimens and media |
US20130060584A1 (en) * | 2011-09-02 | 2013-03-07 | The Travelers Indemnity Company | Systems and methods for customer-driven risk analysis |
US20160140643A1 (en) * | 2014-11-18 | 2016-05-19 | Microsoft Technology Licensing | Multilingual Content Based Recommendation System |
US20160232630A1 (en) * | 2015-02-09 | 2016-08-11 | Legalogic Ltd. | System and method in support of digital document analysis |
US20160357934A1 (en) * | 2014-11-14 | 2016-12-08 | Humana Inc. | Diabetes onset and progression prediction using a computerized model |
US20170017721A1 (en) * | 2015-07-13 | 2017-01-19 | Facebook, Inc. | Generating snippet modules on online social networks |
US20170111506A1 (en) * | 2015-10-14 | 2017-04-20 | Pindrop Security, Inc. | Fraud detection in interactive voice response systems |
US20180129944A1 (en) * | 2016-11-07 | 2018-05-10 | Xerox Corporation | Document understanding using conditional random fields |
US20190163817A1 (en) * | 2017-11-29 | 2019-05-30 | Oracle International Corporation | Approaches for large-scale classification and semantic text summarization |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6778979B2 (en) * | 2001-08-13 | 2004-08-17 | Xerox Corporation | System for automatically generating queries |
US8401841B2 (en) * | 2006-08-31 | 2013-03-19 | Orcatec Llc | Retrieval of documents using language models |
US8788523B2 (en) * | 2008-01-15 | 2014-07-22 | Thomson Reuters Global Resources | Systems, methods and software for processing phrases and clauses in legal documents |
US8473279B2 (en) * | 2008-05-30 | 2013-06-25 | Eiman Al-Shammari | Lemmatizing, stemming, and query expansion method and system |
US20100191731A1 (en) | 2009-01-23 | 2010-07-29 | Vasile Rus | Methods and systems for automatic clustering of defect reports |
US8643648B2 (en) | 2009-03-31 | 2014-02-04 | Patientslikeme, Inc. | Systems, methods, and computer-readable media for context-linked importation of user information |
US20110255794A1 (en) * | 2010-01-15 | 2011-10-20 | Copanion, Inc. | Systems and methods for automatically extracting data by narrowing data search scope using contour matching |
US9830663B2 (en) * | 2012-11-08 | 2017-11-28 | Hartford Fire Insurance Company | System and method for determination of insurance classification and underwriting determination for entities |
US20160103823A1 (en) * | 2014-10-10 | 2016-04-14 | The Trustees Of Columbia University In The City Of New York | Machine Learning Extraction of Free-Form Textual Rules and Provisions From Legal Documents |
US9720901B2 (en) * | 2015-11-19 | 2017-08-01 | King Abdulaziz City For Science And Technology | Automated text-evaluation of user generated text |
US20180011976A1 (en) * | 2016-07-08 | 2018-01-11 | Amino, Inc. | Self-service healthcare platform |
US10380259B2 (en) * | 2017-05-22 | 2019-08-13 | International Business Machines Corporation | Deep embedding for natural language content based on semantic dependencies |
-
2018
- 2018-12-21 EP EP18906514.7A patent/EP3752929A4/en active Pending
- 2018-12-21 WO PCT/US2018/067056 patent/WO2019160608A1/en unknown
- 2018-12-21 US US16/229,261 patent/US20190259104A1/en not_active Abandoned
-
2023
- 2023-09-22 US US18/472,538 patent/US12100056B2/en active Active
Patent Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070282824A1 (en) * | 2006-05-31 | 2007-12-06 | Ellingsworth Martin E | Method and system for classifying documents |
US20080195577A1 (en) * | 2007-02-09 | 2008-08-14 | Wei Fan | Automatically and adaptively determining execution plans for queries with parameter markers |
US20080270268A1 (en) * | 2007-04-27 | 2008-10-30 | Caterpillar Inc. | System and method for service loss analysis and reporting |
US20130054603A1 (en) * | 2010-06-25 | 2013-02-28 | U.S. Govt. As Repr. By The Secretary Of The Army | Method and apparatus for classifying known specimens and media using spectral properties and identifying unknown specimens and media |
US20130060584A1 (en) * | 2011-09-02 | 2013-03-07 | The Travelers Indemnity Company | Systems and methods for customer-driven risk analysis |
US20160357934A1 (en) * | 2014-11-14 | 2016-12-08 | Humana Inc. | Diabetes onset and progression prediction using a computerized model |
US20160140643A1 (en) * | 2014-11-18 | 2016-05-19 | Microsoft Technology Licensing | Multilingual Content Based Recommendation System |
US20160232630A1 (en) * | 2015-02-09 | 2016-08-11 | Legalogic Ltd. | System and method in support of digital document analysis |
US20170017721A1 (en) * | 2015-07-13 | 2017-01-19 | Facebook, Inc. | Generating snippet modules on online social networks |
US20170111506A1 (en) * | 2015-10-14 | 2017-04-20 | Pindrop Security, Inc. | Fraud detection in interactive voice response systems |
US20180129944A1 (en) * | 2016-11-07 | 2018-05-10 | Xerox Corporation | Document understanding using conditional random fields |
US20190163817A1 (en) * | 2017-11-29 | 2019-05-30 | Oracle International Corporation | Approaches for large-scale classification and semantic text summarization |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022135915A1 (en) * | 2020-12-24 | 2022-06-30 | British Telecommunications Public Limited Company | Pre-processing for natural language processing |
CN113505228A (en) * | 2021-07-22 | 2021-10-15 | 上海弘玑信息技术有限公司 | Multi-dimensional text data classification method, training method and device |
CN116150341A (en) * | 2023-04-23 | 2023-05-23 | 之江实验室 | Claim event detection method, computer equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
US20240013315A1 (en) | 2024-01-11 |
WO2019160608A1 (en) | 2019-08-22 |
EP3752929A1 (en) | 2020-12-23 |
US12100056B2 (en) | 2024-09-24 |
EP3752929A4 (en) | 2021-11-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US12100056B2 (en) | Computer-implemented methods, computer-readable media, and systems for identifying causes of loss | |
US20190286704A1 (en) | Learning document embeddings with convolutional neural network architectures | |
Jalbert et al. | Automated duplicate detection for bug tracking systems | |
US20140058763A1 (en) | Fraud detection methods and systems | |
CN110059137B (en) | Transaction classification system | |
CA3165582A1 (en) | Data processing method and system based on similarity model | |
WO2021121252A1 (en) | Comment-based behavior prediction | |
Khodadadi et al. | A natural language processing and deep learning based model for automated vehicle diagnostics using free-text customer service reports | |
Liu et al. | Identifying individual expectations in service recovery through natural language processing and machine learning | |
CN113064883A (en) | Construction method, computer equipment and storage medium of logistics risk control model | |
CA3092332A1 (en) | System and method for machine learning architecture for interdependence detection | |
EP1886206A1 (en) | Test mining systems and methods for early detection and warning | |
US11688012B2 (en) | Asset assessment via graphical encoding of liability | |
CN114969253A (en) | Market subject and policy matching method and device, computing device and medium | |
US20210097605A1 (en) | Poly-structured data analytics | |
CN113850331B (en) | Method, device, equipment and storage medium for detecting abnormality of reimbursement bill | |
US10755184B1 (en) | Predictive machine learning models | |
EP4447413A2 (en) | Machine-learned classification of network traffic | |
CN112651493A (en) | Accident vehicle distinguishing method and device based on joint training model | |
Reddy et al. | Accident analysis and severity prediction of road accidents in United States using machine learning algorithms | |
CN110795537B (en) | Method, device, equipment and medium for determining improvement strategy of target commodity | |
CN114493896A (en) | Data evaluation method, device and equipment based on machine learning and storage medium | |
US20240144050A1 (en) | Stacked machine learning models for transaction categorization | |
US20230038645A1 (en) | Method, electronic device and storage medium for remote damage assessment of vehicle | |
Shi et al. | Mining Consumer Complaints for Recall Management: A Topic Model for Decision Automation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCV | Information on status: appeal procedure |
Free format text: NOTICE OF APPEAL FILED |
|
STCV | Information on status: appeal procedure |
Free format text: APPEAL BRIEF (OR SUPPLEMENTAL BRIEF) ENTERED AND FORWARDED TO EXAMINER |
|
STCV | Information on status: appeal procedure |
Free format text: EXAMINER'S ANSWER TO APPEAL BRIEF MAILED |
|
STCV | Information on status: appeal procedure |
Free format text: ON APPEAL -- AWAITING DECISION BY THE BOARD OF APPEALS |
|
STCV | Information on status: appeal procedure |
Free format text: BOARD OF APPEALS DECISION RENDERED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION |