CN109791627B - Semiconductor device modeling for training deep neural networks using input preprocessing and transformation targets - Google Patents

Abstract

A semiconductor device modeling system and method based on a deep neural network. Training data is collected from measurements of test transistors, including gate and drain voltages, transistor width and length, and drain current measured under the above input conditions. The training data is transformed by an input preprocessing module, which may employ a method of taking the logarithm of the input data or performing a principal component analysis (PCA). When training the deep neural network, instead of using the measured drain current as the fitting target, the converted drain current generated by the target conversion module is used as the fitting target. Derivative, or the logarithm of its derivative. The total error is obtained by comparing the output of the input data through the deep neural network with the converted drain current. By adjusting the weight of the deep neural network during the training period, the total error value is continuously reduced, and the training is finally completed.

Description

Semiconductors for Training Deep Neural Networks Using Input Preprocessing and Transforming Targets Device modeling

【Technical field】

The present invention relates to semiconductor device modeling, and in particular to modeling devices using artificial neural networks.

【Background technique】

A single integrated circuit (IC) may contain a million transistors. Each transistor is typically a metal oxide semiconductor field effect transistor (MOSFET) or a variant thereof formed on a semiconductor substrate. During the IC design process, a netlist or schematic is created to detail the connections between these transistors and other components such as capacitors and resistors. The netlist can then be simulated using a circuit simulator, which uses the device model to simulate the operation of each transistor.

The device model estimates the electrical properties of the transistor, such as drain current as a function of gate and drain voltages. More accurate simulations can use more accurate models to estimate other parameters such as parasitic capacitances to better estimate delays and circuit timing.

An important simulator is the Program for Emulation of Integrated Circuits Emphasis (SPICE), originally developed in 1975 by the University of California, Berkeley. SPICE has been expanded and enhanced since then, and has many variants. The Berkeley short-channel IGFET model (BSIM) is another model that is particularly well suited for small-scale transistor devices.

Test circuits, such as transistors with integrated circuits whose test pads can be manually probed, allow device engineers to manually probe and test these devices to measure current as a function of voltage. Using these test results, device engineers can determine device parameters for use in SPICE or BSIM device models and use these parameters to simulate larger scale ICs. Although these manual measurements are replaced by automated measurements, extracting device parameters for use with SPICE or BSIM models is still time-consuming.

As device sizes shrink, basic first-order device models fail to accurately estimate current for smaller devices. Second-order effects caused by short channel lengths, buried layers, and submicron geometries require new parameters and more complex device modeling equations. More test equipment of different sizes and shapes needs to be added and tested to obtain values for these additional parameters. Automatic measurement equipment allows faster extraction of device model parameters.

As device dimensions continue to shrink, devices with gate lengths of 10 nanometers or less present additional challenges for device modeling as device dimensions approach the atomic dimensions in semiconductor substrates. New semiconductor materials are being used, such as gallium nitride (GaN), gallium arsenide (GaAs) and silicon carbide (SiC), which have different physical properties than silicon. Special devices, such as fin field-effect transistors (FinFETs) and silicon-on-insulator (SOI), have three-dimensional current flow that cannot be accurately modeled using older two-dimensional current models. Measuring the actual current of test devices of different sizes and shapes is critical to creating useful device models.

More recently, artificial neural networks (ANNs) have been used to generate device models and select model parameters. Artificial neural networks are particularly useful for processing large amounts of data in ways that are complex and difficult to define using traditional computer programs. Instead of programming with instructions, training data is fed into the neural network and compared to the expected output, adjustments are made within the neural network, and the training data is processed again and compared to produce further adjustments to the neural network. After many such training cycles, the trained neural network can efficiently process data similar to the training data and expected output. A neural network is an example of machine learning because a neural network learns how to generate the expected output from the training data. Next, real data similar to the training data can be fed into the neural network to process the real-time data.

Figure 1 shows a prior art neural network. Input nodes 102, 104, 106, 108 receive input data I ₁ , I ₂ , _{I 3} , _. O ₂ , O ₃ , ..., O ₄ . In this neural network there are three intermediate layers. Nodes 110, 112, 114, 116, 118, each take input from one or more of input nodes 102, 104, 106, 108, perform some operation such as add, subtract, multiply or more complex operations, and send and outputs to the nodes of the second layer. Second tier nodes 120, 122, 124, 126, 128, 129 also receive multiple inputs, combine these inputs to generate outputs, and send the outputs to third tier nodes 132, 134, 136, 138, 139, which similarly Merge inputs and generate outputs.

The inputs to each layer are usually weighted so that a weighted sum (or other weighted operation result) is produced at each node. Each input on a node is assigned a weight, that weight is multiplied by that input, and all weighted inputs are added, multiplied, or otherwise operated on together by the node to produce the node's output. These weights can be designated as _W31 , _W32 , _W32 , _W33 , . . . , _W41 , etc., and their values adjusted during training. Through trial and error or other training procedures, eventually paths that produce the expected output can be given higher weights, while paths that do not produce the expected output can be assigned smaller weights. The machine learns which paths produce the desired output and assigns high weights to the inputs of those paths.

These weights may be stored in the weight memory 100 . Since neural networks typically have many nodes, many weights are stored in the weight memory 100 . Each weight may require multiple binary bits to represent the range of possible values for that weight. Weights typically require 8 to 16 bits.

Figure 2 shows a transistor device model. The gate voltage Vg and drain voltage Vd are applied to the transistor, while the source voltage Vs is normally grounded. The substrate or bulk voltage Vb can be ground, or another voltage such as reverse bias. The device model uses various parameters to predict the drain current Ids (drain to source current), which is a function of Vg, Vd, Vb, Vs. Other inputs such as temperature T, gate width W and gate length L also affect the predicted drain current, especially when L is very small.

Figure 3 shows the overfitting problem when modeling the device. The measurement data 204 , 206 , 208 are input into a neural network to generate model parameters that best fit the measurement data 204 , 206 , 208 . The model current 202 is the best fit model of the measurement data 204 , 206 , 208 . The measurement data 206, 208 are two outlier data points, which may be the result of some measurement error. Fitting all data points including anomalous measurement data 206 , 208 using a neural network causes model current 202 to spike down to anomalous measurement data 208 , then sharply rise to anomalous data point 206 , and down to measurement data 204 . The abnormal measurement data 206, 208 cause the model current 202 to have a negative conductance around the abnormal data points. Also, model currents 202 that exceed measured data 204 may be unreliable, prone to erratic behavior. Poor scalability.

4A-4B show poor modeling when the model reaches a minimum value. In Figure 4A, the drain current is plotted as a function of drain-source voltage. Training data 214 and test data 212 are measurement data points, the training data 214 is input to the neural network to generate weight values, and the test data 212 is used to test the accuracy of the neural network weight values. 218 is a model generated by inputting different gate voltages Vg. While the accuracy of the model 218 appears to be better at higher currents as shown in FIG. 4A , the accuracy of the model 218 at lower currents as shown in FIG. 4B is poor. The model 218 does not converge at the origin (0,0), but near the origin. Model 218 fails for voltages and currents in the subthreshold region.

Figure 5 shows training a neural network to produce a device model using drain current as a fitting target. The test transistors are measured, and the input voltage, temperature, channel width W, and channel length L are recorded as training data 34, and the measured drain current Ids is recorded as fitting target data 38 corresponding to the input data combination. Neural network 36 receives training data 34 and a current set of weights, and operates on training data 34 to produce a result. The results produced by the neural network 36 are compared to the fit target data 38 by a loss function 42, which produces a loss value that indicates the error of the produced results from the fit target. The loss values produced by the loss function 42 are used to adjust the weights applied to the neural network 36 . By applying the loss function 42 to the training data 34, the weights are iterated many times until a minimum loss value is found, and the final set of weights is used for the transistor model.

Device models are expected to be accurate over a wider range (from subthreshold to strong inversion regions). However, using a neural network leads to the overfitting problem of Figure 3 and the sub-threshold accuracy of Figure 4B. Also, some circuit simulators use model derivatives or slopes such as conductance (Gds) and transconductance (Gm), but model convergence issues may distort the extracted conductance (Gds) and transconductance (Gm) values. The first derivative of the model may have poor accuracy. Monotonicity can be bad. To avoid overfitting and bad monotonicity issues, it may be necessary to limit the size of the hidden layers, which makes the use of deep neural networks difficult. However, shallow neural networks cannot be applied to more complex models if an accurate model is still desired.

It is desirable to have a device model for semiconductor integrated circuits (ICs) that accurately simulates a wide range of currents, including the subthreshold region. The device models produced by the neural network are desirable, but do not have problems with overfitting. A device model capable of accurately modeling conductance (Gds) and transconductance (Gm) values is desirable.

【Description of drawings】

Figure 1 shows a prior art neural network.

Figure 2 shows a transistor device model.

Figure 3 shows the overfitting problem when modeling the device.

Figures 4A-4B show less accurate models near the minimum.

Figure 5 shows training a neural network using the drain current as the target to produce a device model.

FIG. 6 is a schematic diagram of an artificial neural network that converts semiconductor drain currents.

Figure 7 shows a deep neural network that operates on preprocessed inputs and uses a loss function that transforms the target to adjust the weights.

Figure 8 shows using the transformed drain current as a fitting target for a deep neural network to address overfitting in device modeling.

Figures 9A-9B show that using the transformed drain current as a fitting target enables deep neural networks to better model subthreshold transistors.

Figure 10 shows a transistor simulator that obtains models and parameters based on a deep neural network that preprocesses input operations and is fitted to convert drain current.

【Detailed ways】

The present invention relates to improvements in the modeling of semiconductor devices using artificial neural networks. The following description enables one of ordinary skill in the art to make and use the invention provided in the context of a particular application and its requirements. Various modifications to the preferred embodiment will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

FIG. 6 is a schematic diagram of an artificial neural network with the transformed semiconductor device drain current as a fitting target. While the drain current is approximately linear in some ranges, the drain current may be non-linear in some larger ranges. The inventors have recognized that, unlike first-order models commonly used for transistors, the drain current is non-linear. The inventors believe that using drain current as a fitting target for a neural network will create model accuracy issues, overfitting, poor monotonicity and convergence issues for subthreshold currents.

The inventors have found that model generation and resulting model accuracy are facilitated by transforming the target. Instead of using the drain current Ids as a target, the drain current is converted and the converted drain current is used by the loss function 42 to generate a loss function and adjust the weights.

The measured drain current in target data 38 is converted by target conversion module 44 to converted drain current X_Ids. The target conversion module 44 converts the drain current by means of derivation. The conversion process can be the derivation of the drain current with respect to the gate voltage, drain voltage, substrate voltage, or the size and temperature of the transistor. Taking the logarithm of the above derivative is also a conversion method. The loss function 42 calculates the total difference between the neural network output and its expected output, which is the transition drain current X_Ids. The weights applied to the deep neural network 50 are adjusted by some optimization algorithm, such as stochastic gradient descent, so that the total difference calculated by the loss function 42 becomes smaller and smaller. The difference between the transformed drain current and the neural network output, calculated by the loss function 42, may be smaller than the difference between the drain current and the neural network output.

Training data 34 includes drain-to-source voltage Vds, gate-to-source voltage Vgs, substrate-to-source voltage Vbs, temperature, transistor channel width W and length L, and drain measured under the above conditions to the source current Ids. These input voltages and conditions are processed by the input preprocessing module 40 to generate preprocessed input data as input to the deep neural network 50 .

The input preprocessing module 40 can perform various preprocessing on the training data 34, such as finding the natural logarithm of the input voltage: ln(Vgs), ln(Vgs). The input preprocessing module 40 may perform a principal component analysis (PCA) on the training data 34 to obtain the principal variables that most affect the transformed drain current. PCA can detect which input variable has the most influence on the drain current. PCA can use the eigenvectors of the covariance matrix to reduce the variable dimension.

The deep neural network 50 may be a neural network generated by an engineer for a specific application, or a general neural network tuned for a specific application. For example, the number of intermediate or hidden layers in a neural network can be adjusted for a specific application, the type of operations performed in a node and how nodes are connected can be adjusted for certain applications or problems that need to be solved, such as semiconductor device modeling. The shallow neural network models the drain current Ids, but the deep neural network 50 contains a deep neural network with at least 5 intermediate layers. A deep neural network 50 with more intermediate layers would allow better modeling of second-order effects such as buried layers in semiconductor devices and three-dimensional current flow in complex devices.

The initial weights used in the deep neural network 50 can be set to random values within an initial range such as (-1.0 to 1.0). The training data 34 is preprocessed by the input preprocessing module 40, and the preprocessed data is input to the deep neural network 50 when performing training, so that the output of the deep neural network 50 is evaluated.

One way to assess the quality of the output of the deep neural network 50 is to calculate a loss. Loss function 42 can calculate a loss value, which can measure how close the output of the current loop is to the expected result. A mean squared error (MSE) is created by squaring a single output difference (the difference between the output and the expected value) and averaging or summing these squares over all outputs.

The goal of training is to find weight values that make the network output (predicted values) the same or close to the fit target (data). The process is complex, so it is impossible to calculate the weights mathematically. But computers can learn from data. After preprocessing or target transformation, the data is split into input and fitted targets. First the weights are set to random initial values. When an input vector is presented to the neural network, the value is propagated forward through the neural network layer by layer until it reaches the output layer. The output of the neural network is then compared to the fitted target using a loss function. The loss function for a single output is 1/2|y-y'| ² , where y is the neural network output and y' is the fitting target. The loss function E for n input data is the average of the losses for a single input: E=1/2n*∑|y-y'| ² . After determining the loss for n inputs, an optimization algorithm can be used to adjust the weights and minimize the loss. The optimization algorithm repeats this two-stage cycle, forward computation (propagation) and weight update. Forward computation is used to calculate the total loss. In the second stage, optimization methods such as gradient descent are used to update the weights in an attempt to reduce the loss. These cycles are stopped when the total loss falls below an acceptable value.

The loss function generated by the loss function 42 may also include a complexity loss including a weight decay function (to prevent overfitting when adjusting the weight range), and a sparse function (used to improve the structure and regularity). The complexity loss can prevent overfitting of the model, for example including abnormal measurement data 206 , 208 ( FIG. 3 ), because a model that includes abnormal measurement data 206 , 208 is compared to a smoother model that excludes abnormal measurement data 206 , 208 more complicated.

Both the accuracy loss and the complexity loss can be generated as part of the loss function 42, which adjusts the weights for the next training epoch during training. The updated weights are applied to the deep neural network 50, and the training data 34 preprocessed by the input preprocessing module 40 is input again to the deep neural network 50, and the deep neural network 50 generates a new set of loss values, which are compared to the output by the loss function 42 The result is generated with the converted drain voltage. Adjust the weights and recalculate the loss value in multiple loops until the desired end point is reached. The final weights applied to the deep neural network 50 can then be used to construct the final device model.

Figure 7 shows a deep neural network that operates on preprocessed inputs and uses the transformed loss function to fit the target to adjust the weights. The training data 34 contains input conditions to be measured including gate, drain, bulk voltage, temperature, and transistor size, among others. The preprocessed data generated by the processing of the training data 34 by the input preprocessing module 40 is input to the deep neural network 50 . These preprocessing inputs may include the logarithm of the drain-source voltage, the logarithm of the gate-source voltage, and other inputs selected or combined by principal component analysis (PCA).

The deep neural network 50 operates on the preprocessed input data from the input preprocessing module 40 using a set of weights adjusted by the loss function 42 through an optimization algorithm. The deep neural network 50 can operate forward, where each set of preprocessing inputs is operated on by each node (with a weight between -1 and 1) within the deep neural network 50 to produce one or more outputs, which are determined by the loss function 42 Analysis to generate a new set of weights. The new set of weights is passed back to the deep neural network 50 so that while the input of the deep neural network 50 remains fixed, its output changes as the weights change.

The loss function 42 does not compare the neural network output to the measured drain current. Instead, the drain current measured in target data 38 is converted by target conversion module 44 to converted drain current. The loss function 42 compares the converted drain current to the neural network output.

As shown in FIG. 7 , the converted drain current produced by the target converter 44 and used by the loss function 42 to be compared with the output of the deep neural network 50 may be a derivative of the drain current. The conversion of these drain currents may include the derivative of the drain current with respect to the drain-source voltage d(Ids)/d(Vds), the derivative of the drain current with respect to the gate-source voltage d(Ids)/d(Vgs), the drain current Derivative d(Ids)/d(Vbs) with respect to pad-source voltage, d(Ids)/dT of drain current with respect to temperature, d(Ids)/dL of drain current with respect to transistor channel length, and drain The derivative of the pole current with respect to the transistor channel width, d(Ids)/dW.

The conversion of the drain current by the target inverter 44 may also be the logarithm of any of these derivatives. For example, the natural logarithm of the derivative of the drain current with respect to the drain-source voltage ln[d(Ids)/d(Vds)], the logarithm of the derivative of the drain current with respect to the gate-source voltage ln[d(Ids)/d(Vgs )]Wait.

Using a deep neural network 50 with more layers (deeper) allows more accurate modeling of multiple features that may occur in sub-10-nm transistors and 3D transistors such as Fin Field Effect Transistors (FinFETs) and silicon-on-insulator (SOI). The deep neural network 50 provides a framework for modeling more complex future semiconductor processes.

Figure 8 shows using the transformed drain current as a fitting target for a deep neural network to solve the overfitting problem in device modeling. The measurement data 204, 206, 208 are preprocessed by the input preprocessing module 40 and input into the deep neural network 50, the output of the neural network is compared to the transformed drain current through a loss function 42 to generate the best fit measurement data 204 , 206, 208 model parameters. The modeled current 302 is the best fit model of the measurement data 204 , 206 , 208 .

The measurement data 206, 208 are two outlier data points, which may be the result of some error or measurement error. The drain current values at the data points of the measurement data 206 , 208 are not significantly different from the drain currents of the other measurement data 204 . However, taking the derivative of the drain current increases the error due to the loss function 42 . Although the current values of the data points do not differ much, the slope of the line connecting the data points has a sharp jump near the point of poor fit of the measured data 206, 208 (see line 202 in Figure 3). Therefore, using the converted drain current as the fitting target can enlarge the error produced by the loss function 42 . The regularization method may not have a significant effect on the small error of the drain current, but has a significant effect on the fitting target value increased by the derivation method.

Despite the abnormal measurement data 206, 208, a curve-smoothed model 302 is obtained. There is no negative conductance around outlier data points. Also, the modeled current 302 beyond the measurement data 204 is reliable. Extensibility is good.

Figures 9A-9B show that using the transformed drain current as a fitting target enables deep neural networks to better model the subthreshold region, Figure 9A.

In Figure 9A, the measured drain current is plotted as a function of drain-source voltage. Training data 214 and test data 212 are measurement data points, training data 214 is preprocessed and input to deep neural network 50 to generate weights, and test data 212 is transformed by target transformation module 44 and used to test the accuracy of the neural network weights.

Model 308 is generated for different gate voltages Vg. The accuracy of the model 308 is good for the larger currents in Figure 9A, and also for the smaller currents in Figure 9B. Model 308 converges at the origin (0,0). Model 308 applies to voltages and currents in the subthreshold region.

By targeting the conversion of the drain current rather than the drain current itself, a model 308 that covers a wider range of input voltages can be derived. Models in the subthreshold region are more accurate and converge at the origin.

FIG. 10 shows a transistor simulator based on a deep neural network model based on preprocessing input operations and with the conversion drain current as the fitting target and parameters obtained when the output of the deep neural network 50 best matches the converted drain current. A set of weight values to which the final set of weights can be applied to the deep neural network 50 . Simulation input data 54 includes voltage, temperature, and transistor width and length input by the design engineer during the circuit design process. The simulation input data 54 is operated by the input preprocessing module 40 to preprocess the input data to obtain preprocessed inputs X1, X2, X3, X4, . . . For example, the logarithm of the gate and drain voltages may be obtained by the input preprocessor 40 and input to the deep neural network 50 .

The deep neural network 50 generates the converted drain current X_Ids according to the preprocessing input and the final weights. The operation performed by inverse target converter 58 is the inverse conversion of target converter 44 (FIG. 6). For example, when target converter 44 produces the derivative of the drain current with respect to the gate voltage, inverse target converter 58 may integrate the converted drain current over the gate voltage.

The inverse target converter 58 may use Riemann sums, Newton-Cortes formulas, or use linear or polynomial interpolation for integration. When the target conversion module 44 performs the logarithm of the derivative of the drain voltage, the inverse target conversion module 58 may generate an exponential function that converts the drain current and then integrate. Reverse target conversion module 58 generates the drain current value prior to conversion by target conversion module 44 . This is the simulated drain current predicted by simulator 60 .

During the simulation, the loss function 42 and target transformation module 44 are not needed because the final weights remain constant and are not adjusted. The simulator 60 may consist of an input preprocessor 40 , a deep neural network 50 and an inverse object transformer 58 . The size of the deep neural network 50 can be reduced, for example, by removing nodes with a weight of zero. When used inside the simulator 60, the deep neural network 50 may be a forward neural network. In larger scale circuit simulators, SPICE, BSIM or other device models may be replaced by simulator 60 . For each transistor in the simulation, the larger circuit simulator described above will generate the input voltage, look up transistors W and L, read the specified temperature for the simulation, and then invoke the simulator 60 to simulate the drain current of that transistor.

By using a deep neural network 50 with a target transformation module 44 and an input preprocessing module 40, the time and labor required to develop a device model can be significantly reduced. Measurement data can be obtained from devices on test chips fabricated using the new process and used as training data 34 (Vgs, Vds, . . . ) and fit target data 38 (Ids). The deep neural network 50 may operate forward and backward to adjust the weights until the loss function 42 finds an acceptable low loss or minimum value. The final weights can then be applied to the deep neural network 50 and simulated along with the loss function 42 and the inverse target conversion module 58 for transistors fabricated by the new process.

[Other Embodiments]

Several other embodiments are also contemplated by the inventors. For example, object converter 44, input preprocessor 40, and inverse object converter 58 may share the same computing hardware, or each may have dedicated hardware. Transforming the drain current may be the derivative of the drain current, the logarithm of these derivatives, the conductance g(ds), the transconductance g(m), the logarithm of the conductance or transconductance, or other transformations of the drain current. Several of these transformations can be tested to find the best one to use as the objective that yields the lowest loss function. Similarly, input preprocessor 40 may preprocess some or all of the input in various ways. The logarithm can be the natural logarithm, or base 10 logarithm, or use other bases. Various combinations of transformation or preprocessing functions can also be substituted.

Some embodiments may not use all components. Additional components can be added. The loss function 42 may use various error/loss generators, such as a weight decay term that prevents weights from growing too large over multiple training optimization loops, a sparsity penalty that encourages nodes to zero their weights so that only a small fraction of nodes are Use effectively. The remaining small subset of nodes are the most relevant. While various loss and cost functions have been described in the principles, many alternatives, combinations and variations are possible. Other variations and types of loss or cost terms can be added to the loss function 42 . The values of the relative scaling factors for the different cost functions can be adjusted to balance the effects of the individual functions.

Floating-point values can be converted to fixed-point or binary values. Although binary-valued weights have been shown, various encodings can be used, such as two's complement, Huffman encoding, truncated binary encoding, etc. The number of binary bits required to represent the weight value may refer to the number of bits regardless of the encoding method, whether it is binary encoding, gray code encoding, fixed point, offset, or the like.

Weights can be restricted to a range of values. Although a range of -1 to 1 has been described, the range does not necessarily have to include 0, such as a range of 512 to 1. The weight value can be offset to fit a binary range, such as a weight in the range 10511 to 10000, which can be stored as a 9-bit binary word to which an offset of 10000 is added to produce the actual weight value. The range can be adjusted during optimization. The offsets may be stored or may be hardwired into the logic of the deep neural network 50 .

Numerous variations are possible for the training routine for running deep neural network 50 . The optimization can first determine the number of hidden or intermediate layer nodes, and then proceed to optimize the weights. Weights The placement or connectivity of nodes can be determined by zeroing some weights to sever connections between nodes. When the structure is optimized, the sparse cost can be used in the initial loop of optimization, but when the weight values are fine-tuned, the sparse cost is not applicable in the later optimization loop. Hidden layers within deep neural network 50 may be trained using a sigmoid function. Lookup tables can be used to implement more complex functions instead of using an arithmetic logic unit (ALU) to speed up processing. The activation function may be different for each node, such as sigmoid, tanh, and relu.

The process of loss reduction may be different for different applications and training data. A wide variety of structures, different numbers, and hidden layer arrangements can be used for deep neural network 50 . The activation function may be different for each node, such as sigmoid, tanh, and relu. For specific applications and modeling of semiconductor processes, suitable neural networks may be some specific neural networks, a deep neural network 50 with a specific structure, or a general neural network 50 as a starting point for optimization. The deep neural network 50 may have at least 7 intermediate layers and have at least ten thousand weights.

Autoencoders, automax and softmax classifiers, and other kinds of layers can be plugged into neural networks. The entire optimization process can be repeated multiple times, for example, for different initial conditions, such as quantized floating-point values or other parameters with different numbers of bits, different precisions, different scaling factors, etc. Endpoints can be set for various combinations of conditions, such as desired final accuracy, accuracy-hardware cost product, target hardware cost, etc.

While the actual cost of a deep neural network 50 depends on a variety of factors, such as number of nodes, weights, interconnects, controls, and interfaces, the inventors approximated the cost as proportional to the sum of the weights. The total number of binary bits used to represent all the weights in the deep neural network 50 is a measure of hardware cost, even if only an approximation. The gradient or slope of the hardware complexity cost gradient can be used. The gradient values can be scaled and changed before or after the comparison.

IC semiconductor manufacturing processes may vary in many ways. Photomasks can be used with various special machines and processes, including direct writing to burn off metallization rather than photoresist. Various combinations of diffusion, oxide growth, etching, deposition, ion implantation, and other fabrication steps can produce the resulting pattern on an IC controlled by a photomask. Although modeling transistors has been described, and leakage currents in particular, other currents can be modeled, such as diode currents, substrate leakage currents, etc., and can be used for modeling of other devices such as capacitors, resistors, and the like.

Deep neural network 50, loss function 42, target conversion module 44, inverse target conversion module 58, and other components may be implemented in various techniques using various combinations of software, hardware, firmware, programs, modules, functions, and the like. The final product, the deep neural network 50 with final weights as well as the input preprocessing module 40 and the inverse target transformation module 58, can be implemented in an application specific integrated circuit (ASIC) or other hardware for when the simulator 60 is used to simulate large circuits Increase processing speed and reduce power consumption.

The Background of the Invention section may contain background information about the problem or environment of the invention, rather than describing the prior art by others. Accordingly, the inclusion of material in the Background section is not an admission by the applicant of prior art.

Any method or process described herein is machine-implemented or computer-implemented and is intended to be performed by a machine, computer or other device, not only by a human without such machine assistance. The generated tangible results may include reports or other machine-generated displays on display devices, such as computer monitors, projection devices, audio-generating devices, and related media devices, and may include hard-copy printouts that are also machine-generated. Computer control of other machines is another tangible result.

Any of the advantages and benefits described may not apply to all embodiments of the invention. When reciting the term "means" in a claim element, applicants intend for the claim element to fall within the provisions of 35 USC Chapter 112, paragraph 6. The term or terms preceding the term "means" is intended to facilitate reference to claim elements and is not intended to convey structural limitations. Such means-plus-function claims are intended to cover not only the structures described herein as performing the function and their structural equivalents, but also equivalent structures. For example, although nails and screws have different configurations, they are equivalent structures because they both perform the function of fastening. Claims that do not use the word "device" do not fall under 35 USC Chapter 112, paragraph 6. The signal is usually an electrical signal, but can be an optical signal, such as a signal that can be carried over fiber optic lines.

The foregoing descriptions of embodiments of the present invention have been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. It is intended that the scope of the invention be limited not by this detailed description, but by the appended claims.

Claims (18)

1. A semiconductor device modeling system, comprising: a deep neural network comprising a plurality of nodes, each node scaling its input using a weight, the resulting node output is transmitted to other nodes in the plurality of nodes; an input module for receiving training data, the training data representing the gate voltage and the drain voltage of the transistor, the training data further including the width and length of the transistor; an input preprocessing module that receives the training data from the input module, the input preprocessing module converts the training data into preprocessed training data; wherein the preprocessed training data is applied to the input of the deep neural network; a target input module that receives target data representing the measured drain current of the transistor when the gate and drain voltages represented by the training data are applied to the transistor, and the a transistor having a transistor width and a transistor length represented by the training data; a target conversion module that receives the training data from the target input module, the target conversion module converts target data representing drain current into a converted drain current value; a loss function generation module that compares the transformed drain current generated by the target transformation module with the output generated by the deep neural network based on the preprocessed training data and using the weights, wherein the The loss function generation module minimizes the loss function value between the converted drain current value and the output generated by the deep neural network by adjusting the weight of the neural network; wherein a plurality of the training data and a plurality of the target data are input and processed during training of the deep neural network to generate a plurality of weights and loss function values; Wherein, after the training is completed, the last set of weights is selected, and the last set of weights generates the minimum loss function value; wherein the last set of weights defines a device model of the transistor that is simulated using the deep neural network; wherein the target conversion module converts the training data representing the drain current into a derivative of the drain current; The derivative of the drain current is the transformed drain current value, and is also the training target of the deep neural network evaluated by the loss function generation module. 2. The semiconductor device modeling system of claim 1, wherein the derivative of the drain current is a derivative with respect to the gate voltage. 3. The semiconductor device modeling system of claim 1, wherein the derivative of the drain current is a derivative with respect to the drain voltage. 4. The semiconductor device modeling system of claim 1, wherein the derivative of the drain current is a derivative with respect to transistor size. 5. The semiconductor device modeling system of claim 1, wherein the training data further comprises temperature: wherein the derivative of the drain current is a derivative with respect to the temperature. 6. The semiconductor device modeling system of claim 1, wherein the target conversion module converts training data representing drain current into a logarithm of a derivative of the drain current; wherein the logarithm of the derivative of the drain current is the transformed drain current value and is also the training target of the deep neural network evaluated by the loss function generation module. 7. The semiconductor device modeling system of claim 1 , wherein during simulation, the last set of weights is applied to the deep neural network, and simulation training data representing simulated voltages is provided by the input preprocessing module performing preprocessing and then processing by the deep neural network to generate a simulation output by using the final weight values; an inverse target conversion module that receives the simulation output from the deep neural network during simulation and generates a simulated value of drain current, the operation of the inverse target conversion module is the inverse conversion of the target conversion module operate; Thus, simulated values of drain current are generated from the deep neural network using the last set of weights. 8. The semiconductor device modeling system of claim 7, wherein the inverse target conversion module includes an integration module that integrates the simulation output of the deep neural network over a range of voltages to generate The simulated value of the drain current. 9. The semiconductor device modeling system of claim 1, wherein the input preprocessing module generates a logarithm of gate voltage or a logarithm of drain voltage as preprocessing training data as input to the deep neural network , Thus, a logarithmic voltage input is applied to the deep neural network. 10. The semiconductor device modeling system of claim 1, wherein the input preprocessing module performs a principal component analysis (PCA) on the training data to select master data, wherein the master data is applied to a deep neural network input The preprocessed training data, Thus, PCA is performed and then applied to the deep neural network. 11. A computer-implemented method for simulating an analog transistor, comprising: receiving input data representing gate and drain voltages applied to the analog transistor by a circuit simulator, the input data further including a transistor width and a transistor length of the analog transistor; preprocessing the input data to generate preprocessed input data by generating the logarithm of the gate voltage or the logarithm of the drain voltage; applying the preprocessed input data as input to a deep neural network; the deep neural network includes a set of weights and inputs the preprocessed input data to the deep neural network to produce a neural network output; integrating the neural network output over a gate voltage or a drain voltage to generate a drain current value for the analog transistor; outputting the drain current value to the circuit simulator, the circuit simulator using the drain current value to simulate operation of the transistor in a circuit; wherein the deep neural network receives the logarithm of the voltage and generates a derivative of the drain current value during simulation. 12. The computer-implemented method of claim 11, wherein the deep neural network has at least seven intermediate layers, wherein the deep neural network has at least ten thousand weights. 13. The computer-implemented method of claim 11, wherein the neural network output is a conductance value. 14. The computer-implemented method of claim 11, wherein the neural network output is a transconductance value. 15. A non-transitory computer-readable medium for storing computer-executable instructions that, when executed by a processor, implement a method, the method comprising: applying initial weights to connections between nodes in a neural network, weights specifying the strength of connections between nodes in said neural network; receiving input data representing gate and drain voltages applied to a test transistor, the input data further including a transistor width and a transistor length of the test transistor; preprocessing the input data to generate preprocessed input data; executing a training routine to input the preprocessed input data to the neural network; receiving target data representing a drain current measured on the test transistor when the gate voltage and the drain voltage are applied to the test transistor during testing; converting the target data into conversion target data; generating a loss function based on a comparison of the transformation target data with the neural network output when the preprocessed input data is applied to the neural network input by the training routine; to generate updated weights by adjusting the initial weights using the loss function; When the target endpoint has not been reached, applying the update weights to the neural network, performing another iteration, and updating the update weights; When the target end point is reached, output the neural network and the update weight as a model of the test transistor; wherein preprocessing the input data to generate the preprocessing input data further comprises: generating the logarithm of the gate voltage or generating the logarithm of the drain voltage; where the logarithm of the voltage is applied as the input to the neural network. 16. The non-transitory computer-readable medium of claim 15, wherein converting the target data to conversion target data further comprises: The derivative of the drain current is generated as the conversion target data. 17. A non-transitory computer-readable medium for storing computer-executable instructions that, when executed by a processor, implement a method, the method comprising: applying initial weights to connections between nodes in a neural network, weights specifying the strength of connections between nodes in said neural network; receiving input data representing gate and drain voltages applied to a test transistor, the input data further including a transistor width and a transistor length of the test transistor; preprocessing the input data to generate preprocessed input data; executing a training routine to input the preprocessed input data to the neural network; receiving target data representing a drain current measured on the test transistor when the gate voltage and the drain voltage are applied to the test transistor during testing; converting the target data into conversion target data; generating a loss function based on a comparison of the transformation target data with the neural network output when the preprocessed input data is applied to the neural network input by the training routine; to generate updated weights by adjusting the initial weights using the loss function; When the target endpoint has not been reached, applying the update weights to the neural network, performing another iteration, and updating the update weights; When the target end point is reached, output the neural network and the update weight as a model of the test transistor; The converting the target data into conversion target data further includes: The logarithm of the derivative of the drain current is generated as the conversion target data. 18. A non-transitory computer-readable medium for storing computer-executable instructions that, when executed by a processor, implement a method, the method comprising: applying initial weights to connections between nodes in a neural network, weights specifying the strength of connections between nodes in said neural network; receiving input data representing gate and drain voltages applied to a test transistor, the input data further including a transistor width and a transistor length of the test transistor; preprocessing the input data to generate preprocessed input data; executing a training routine to input the preprocessed input data to the neural network; receiving target data representing a drain current measured on the test transistor when the gate voltage and the drain voltage are applied to the test transistor during testing; converting the target data into conversion target data; generating a loss function based on a comparison of the transformation target data with the neural network output when the preprocessed input data is applied to the neural network input by the training routine; to generate updated weights by adjusting the initial weights using the loss function; When the target endpoint has not been reached, applying the update weights to the neural network, performing another iteration, and updating the update weights; When the target end point is reached, output the neural network and the update weight as a model of the test transistor; The method further includes: after said target endpoint has been reached; applying update weights to the neural network; receiving input data representing gate and drain voltages applied to an analog transistor, the input data further including a transistor width and a transistor length of the analog transistor; preprocessing the input data to generate preprocessed input data; inputting the preprocessed input data to the neural network and operating on the preprocessed input data using the neural network and the update weights to produce a neural network output; The neural network output is integrated to generate an analog drain current for the analog transistor.

Images