CN114462772A - Semiconductor manufacturing scheduling method, system, and computer-readable storage medium - Google Patents

Abstract

The invention discloses a semiconductor manufacturing scheduling method, a system and a computer readable storage medium, wherein the method comprises the following steps: determining a mathematical optimization model according to the data scale; generating a first scheduling sequence according to the mathematical optimization model, and performing iterative computation on the first scheduling sequence under a constraint condition through the mathematical optimization model to obtain a second scheduling sequence; determining an optimal strategy based on the operation result of each training action, determining a target total strategy according to the optimal strategy corresponding to each state parameter, and performing simulation scheduling according to the target total strategy to obtain a third scheduling sequence; and performing scheduling operation according to the third scheduling sequence. The invention realizes the rapid target optimization of semiconductor manufacturing scheduling, effectively and reasonably distributes production capacity, improves the resource utilization rate and reduces the production cost.

Description

Semiconductor manufacturing scheduling method, system, and computer-readable storage medium

technical field

The present invention relates to the technical field of semiconductor manufacturing, and in particular, to a semiconductor manufacturing scheduling method, system, and computer-readable storage medium.

Background technique

Semiconductor manufacturing is one of the complex industrial manufacturing chains that typically include four production stages: wafer fabrication, wafer sorting, packaging, and product testing. Among them, wafer fabrication is a complex job, and the fabrication of machines and silicon wafers requires hundreds of process steps. A small part of the product exists in hundreds or thousands of processes in the factory. Therefore, it is necessary to create complex capacity allocation and job scheduling problems, that is, scheduling problems are closely related issues in semiconductor manufacturing production. The scheduling problem generally refers to the pipeline processing of n workpieces on m machines. Each workpiece takes different time to run on each machine, and each machine can only process one workpiece at a time. The processing sequence on each machine and the start time of each process can minimize the maximum completion time or optimize other indicators.

At present, an existing semiconductor production scheduling method uses evolutionary laws to construct intelligent optimization algorithms such as genetic algorithms for querying. However, this method needs to construct a sequence population and perform optimization iterations in the population. Each individual is calculated separately. When the population is larger, the amount of calculation is also larger, so that it takes a lot of time to solve once.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a semiconductor manufacturing scheduling method, system and computer-readable storage medium, which can achieve effective and reasonable distribution of production capacity, improve resource utilization, and reduce production costs.

In order to achieve the above object, the present invention proposes a method for scheduling semiconductor manufacturing, the method comprising the following steps:

Obtain the data scale of semiconductor manufacturing scheduling, and determine a mathematical optimization model according to the data scale;

A first scheduling sequence is generated according to the mathematical optimization model, and a second scheduling sequence is obtained by iteratively calculating the first scheduling sequence through the mathematical optimization model under constraints. The sequence sequence is a scheduling sequence sequence with the largest fitness value under the constraint condition, and the constraint condition includes the maximum number of iterations;

determining all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traversing each of the state parameters, and determining all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies;

Run the training actions in each of the traversal initial strategies, determine the optimal strategy based on the running results of each of the training actions, determine the overall target strategy according to the optimal strategy corresponding to each of the state parameters, and perform the strategy according to the overall target strategy. Simulate production scheduling to obtain the third scheduling sequence;

The scheduling operation is performed according to the third scheduling sequence.

Further, in the above semiconductor manufacturing scheduling method, the data scale includes the number of products to be processed.

Further, in the above-mentioned semiconductor manufacturing scheduling method, the step of obtaining the second scheduling sequence by iteratively calculating the first scheduling sequence through the mathematical optimization model under constraints specifically includes:

When the data size does not exceed a preset threshold, two random integers are generated, where the two random integers are different integers between 1 and N, where N is the length of the first scheduling sequence;

Interchange the numerical values of the serial number positions corresponding to the two random integers in the first scheduling sequence to obtain a first comparison sequence;

Calculate and compare the fitness value of the first scheduling sequence sequence with the fitness value of the first comparison sequence sequence; when the fitness value of the first comparison sequence sequence is greater than the fitness value of the first scheduling sequence The first comparison sequence sequence repeats the steps of the value exchange until the maximum number of iterations in the constraint condition is satisfied, and a second sequence sequence sequence is obtained, and the second sequence sequence sequence is the one with the largest adaptation value under the constraint condition. Schedule a sequential sequence.

Further, in the above-mentioned semiconductor manufacturing scheduling method, the step of obtaining the second scheduling sequence by iteratively calculating the first scheduling sequence through the mathematical optimization model under constraints specifically includes:

When the data size exceeds a preset threshold, perform roulette screening, crossover operation and mutation operation on the first scheduling sequence under constraints, where the constraints include the maximum number of iterations including the maximum number of iterations. The number of iterations, the number of individuals in the population, the probability of crossover and the probability of mutation;

The second scheduling sequence is determined according to the multiple preferred sequences, and the second scheduled sequence is the preferred sequence with the largest fitness value among the multiple preferred sequences.

Further, in the above-mentioned semiconductor manufacturing scheduling method, the step of performing roulette screening on the first scheduling sequence under constraints specifically includes:

Calculate the fitness values of a plurality of random sequence sequences in the first scheduling sequence sequence respectively, and construct a normalized interval according to the ratio of the fitness value of each random sequence sequence to the sum of the fitness values of all random sequence sequences. The normalized interval includes multiple sub-intervals corresponding to each fitness value, and the value range of the normalized interval is [0, 1];

generating multiple random numbers that are different between 0 and 1, the number of the random numbers is the same as the number of multiple random sequence sequences;

In the normalized interval, the sub-intervals into which each random number falls are determined, and the corresponding random sequence sequence is selected according to the fitness value of the sub-interval that falls into, and a plurality of selected random sequence sequences are obtained.

Further, in the above-mentioned semiconductor manufacturing scheduling method, the step of performing a crossover operation on the first scheduling sequence sequence under constraints specifically includes:

Determine the random sequence sequence to be crossed according to the crossover probability and the multiple random sequence sequences after screening, and pair the random sequence sequences to be crossed in pairs to obtain multiple pairs of sequence groups, each pair of sequence groups includes two random sequence sequences sequence;

Generate a random integer between 1 and N, where N is the length of the first scheduling sequence, and exchange the forward or reverse values of the two random sequence sequences in the sequence group from the random integer position , to obtain multiple random sequence sequences after crossover.

Further, in the above-mentioned semiconductor manufacturing scheduling method, the step of performing a mutation operation on the first scheduling sequence sequence under constraints specifically includes:

Determine the random sequence sequence to be mutated according to the mutation probability and multiple random sequence sequences after crossover;

Generate two random integers between 1 and N, where N is the length of the first scheduling sequence, and randomly sort the subsequences between the corresponding positions of the two random integers in the random sequence to be mutated, Obtain multiple mutated random sequence sequences;

A preferred sequence is calculated according to the plurality of mutated random sequence sequences, and the preferred sequence is the sequence with the largest fitness value among the plurality of mutated random sequence sequences.

Further, in the above-mentioned semiconductor manufacturing scheduling method, the training action includes the delivery sequence, the customer level, the arrival time of the products from the buffer zone, and the initial sequence of the products.

In addition, the present invention also provides a semiconductor manufacturing scheduling system for realizing the above-mentioned semiconductor manufacturing scheduling method, including:

a first determining unit, used for determining a mathematical optimization model according to the data scale of semiconductor manufacturing scheduling;

a first computing unit, configured to generate a first scheduling sequence according to the mathematical optimization model, and obtain a second scheduling sequence by iteratively calculating the first scheduling sequence under constraints by the mathematical optimization model , the second scheduling sequence is a scheduling sequence with a maximum fitness value under a constraint condition, and the constraint condition includes a maximum number of iterations;

The second determining unit is used to determine the priority of the target attribute of the product to be processed;

The second computing unit is configured to determine all the state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traverse each of the state parameters, and determine all the state parameters corresponding to the traversed state parameters based on each of the initial strategies Traversing the initial strategy; and running the training actions in each of the traversing initial strategies, determining the optimal strategy based on the running results of each of the training actions, determining the overall target strategy according to the optimal strategy corresponding to each of the state parameters, and According to the general strategy of the target, the simulation scheduling is carried out, and the third scheduling sequence is obtained;

The scheduling unit is used for scheduling operations according to the third scheduling sequence.

In addition, the present invention also provides a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to realize the above-mentioned semiconductor manufacturing scheduling method.

Through the combination of the global search ability of the intelligent algorithm and the local optimization skill of the reinforcement learning algorithm, the invention realizes the rapid target optimization of the semiconductor manufacturing scheduling, effectively and reasonably allocates the production capacity, improves the resource utilization rate, and reduces the production cost.

Description of drawings

1 is a flowchart of a semiconductor manufacturing scheduling method according to an embodiment of the present invention;

Fig. 2 is the schematic diagram of the fitness value calculation and estimation of the scheduling sequence sequence in the present invention;

Fig. 3 is the schematic diagram of the first mathematical optimization model in the present invention;

Fig. 4 is the schematic diagram of the second mathematical optimization model in the present invention;

Fig. 5 is the scene schematic diagram of the semiconductor manufacturing scheduling method in the present invention;

Fig. 6 is the principle schematic diagram of reinforcement learning training in the present invention;

Fig. 7 is the schematic diagram of the strategy Q value table in the reinforcement learning training of the present invention;

8 is a schematic flowchart of the training of a deep regression model in the reinforcement learning training of the present invention;

9 is a schematic structural diagram of a semiconductor manufacturing scheduling system of a semiconductor manufacturing scheduling method according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of the first computing unit in FIG. 9 .

Detailed ways

The present embodiment takes the semiconductor manufacturing scheduling method and system as an example, and the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.

A semiconductor manufacturing scheduling method provided by an embodiment of the present invention includes the following steps:

Obtain the data scale of semiconductor manufacturing scheduling, and determine a mathematical optimization model according to the data scale; generate a first scheduling sequence sequence according to the mathematical optimization model, and use the mathematical optimization model to perform the first scheduling under constraints The sequence sequence is iteratively calculated to obtain a second sequence sequence sequence, and the second sequence sequence sequence is a sequence sequence sequence with the largest fitness value under the constraint condition, and the constraint condition includes the maximum number of iterations; The state parameters, and the training actions of the second scheduling sequence sequence, traverse each of the state parameters, and determine all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies; run each of the traversal initial strategies. For training actions, the optimal strategy is determined based on the running results of each of the training actions, the overall target strategy is determined according to the optimal strategy corresponding to each of the state parameters, and simulation scheduling is performed according to the overall target strategy to obtain a third schedule Sequence sequence; the scheduling operation is performed according to the third scheduling sequence sequence. Through the combination of the global search ability of the intelligent algorithm and the local optimization skill of the reinforcement learning algorithm, the invention realizes the rapid target optimization of the semiconductor manufacturing scheduling, effectively and reasonably allocates the production capacity, improves the resource utilization rate, and reduces the production cost.

Please refer to FIG. 1 to FIG. 6 , a semiconductor manufacturing scheduling method provided by an embodiment of the present invention specifically includes the following steps:

Step S11: perform simulation initialization on the semiconductor manufacturing scene;

In the specific implementation, during semiconductor manufacturing and processing, such as scheduling the entire process from ingot to silicon wafer, the real environment can be abstracted into an entrance for releasing products, an exit for statistical completed products, and two for storing products. buffers and individual processing machines. The entrance is the generation module of the order, the exit is the confluence of completed products, and each idle machine in different factory areas obtains suitable products from the same buffer. It can be understood that the various components and quantities of each scene may be inconsistent.

Step S12: obtaining the data scale of semiconductor manufacturing scheduling, and determining a mathematical optimization model according to the data scale;

Please refer to FIG. 2 , in the specific implementation, the present invention achieves the optimal strategy of semiconductor scheduling through two-level optimization, wherein, the first-level optimization is to try to calculate the size of the fitness value corresponding to each sequence of semiconductor scheduling through an intelligent algorithm. To find a better sequence, the calculation amount of the intelligent algorithm is related to the data scale of semiconductor manufacturing scheduling. Therefore, it is first necessary to obtain the data scale for evaluating semiconductor manufacturing scheduling, and determine the mathematical optimization model according to the data scale. In this embodiment, the data scale is the number of products to be processed. Of course, the data scale may also be other conditional data such as the total product processing time. In the semiconductor production process, the quantity of a batch of products to be processed is determined in advance according to the production plan, mainly according to the short-term production plan or the medium-term production plan. Before the scheduled production, the quantity of 500 products is determined in advance. According to the different quantities of the products, the calculation amount of the adaptive value F1 of the scheduling calculation is different, and the applied mathematical optimization model is also different. Specifically, according to the experience Or set a threshold for big data. If the data size does not exceed the threshold, the first mathematical optimization model is selected to be applied, and if the data size exceeds the threshold, the second mathematical optimization model is selected to be applied.

That is, the step of determining the mathematical optimization model according to the data scale specifically includes:

When the data size does not exceed the preset threshold, the first mathematical optimization model is selected to be applied; when the data size exceeds the preset threshold, the second mathematical optimization model is selected to be applied.

In this embodiment, the threshold is 500, the first mathematical optimization model is a hill-climbing algorithm model, and the second mathematical optimization model is a genetic algorithm model. That is, in this embodiment, when the number of products to be processed does not exceed 500, the hill-climbing algorithm model is selected to be applied; when the number of products to be processed exceeds 500, the genetic algorithm model is selected to be applied. In this way, an appropriate mathematical optimization model can be selected according to different data scales and calculations, so that model optimization and production scheduling are the fastest and most effective.

Step S13: generating a first scheduling sequence according to the mathematical optimization model;

During specific implementation, when the data scale does not exceed a preset threshold, the first mathematical optimization model is selected to be applied. In this embodiment, the first mathematical optimization model is a hill-climbing algorithm model, the data scale is the number of products to be processed, and a first scheduling sequence is generated according to the number of products to be processed, and the first scheduling sequence is The length is the number of products to be processed. Specifically, when the hill-climbing algorithm model is selected to be applied, a first scheduling sequence P ₀ of length N is generated according to the number N of products to be processed, and the first scheduling sequence P ₀ is a random sequence.

When the data size exceeds a preset threshold, the second mathematical optimization model is selected to be applied. In this embodiment, the second mathematical optimization model is a genetic algorithm model, the data scale is the number of products to be processed, and a first scheduling sequence is generated according to the number of products to be processed and the number of individuals in the population, and the first scheduling sequence is generated. A scheduling sequence sequence includes a plurality of random sequence sequences, the number of the random sequence sequences is the number of individuals in the population, and the length of the random sequence sequence is the number of products to be processed. Specifically, when choosing to apply the genetic algorithm model, according to the number of products to be processed N and the number of individuals m in the population, m random sequence sequences L ₁₁ , L ₁₂ , . . . , L _1m of length N are generated; denote i=1.

Step S14: Iteratively calculates the first scheduling sequence under constraints by using the mathematical optimization model to obtain a second scheduling sequence, where the second scheduling sequence is the one with the largest fitness value under the constraints. scheduling a sequential sequence, the constraints include a maximum number of iterations;

In the specific implementation, please refer to FIG. 3, when the data scale does not exceed a preset threshold, the first mathematical optimization model is a hill-climbing algorithm model, and at this time, the constraint condition is the maximum number of iterations n, the maximum number of iterations n is the number of times estimated based on experience and target accuracy. After generating the first scheduling sequence P ₀ of length N, for example, the first scheduling sequence P ₀ is 2-3-1-7-6-4-8-5; , and then exchange the numerical values corresponding to the serial _number positions of the two random integers in P ₀ and record it as P ₁ . For example, two random integers 3 and 4 are generated. The numerical values at the 4 position are interchanged, and the first comparison sequence P ₁ is obtained as 2-3-7-1-6-4-8-5. Next, the fitness value F ₀ of the first scheduling sequence P ₀ and the fitness value F ₁ of the first comparison sequence P ₁ are calculated and compared.

If F ₀ F ₁ , it means that the first comparison sequence P ₁ is not superior to the first scheduling sequence P ₀ . At this time, it is necessary to re-execute the numerical exchange steps of the above two random integer positions until the first comparison sequence P 1 is found to be better than the first sequence sequence P 0 . The first comparison sequence P ₁ of the sequence sequence P ₀ is scheduled.

If F ₀ <F ₁ , it means that the first comparison sequence sequence P ₁ is better. At this time, continue to select and generate two random integers between 1 and N, and then set the values in P ₁ corresponding to the sequence numbers of the two random integers. Swap, denoted as P ₂ , for example, two random integers 6 and 7 are generated. At this time, the values at the 6th and 7th positions of the first sequence sequence P ₁ are interchanged, and the second sequence sequence P ₂ is obtained as 2 -3-7-1-6-8-4-5. Next, the fitness value F 1 of the first comparison sequence P ₁ and the fitness value F ₂ of the _second comparison sequence P ₂ are calculated and compared. . . By analogy, two random integers between 1 and N are selected to be generated, and then the numerical values corresponding to the serial numbers of the two random integers in Pi are exchanged, and recorded as P _{i +1} , until the maximum number of iterations n ( i>n), the iteration is stopped, that is, the second scheduling sequence sequence P _n is obtained by calculation, and the second scheduling sequence sequence P _n is the scheduling sequence sequence with the largest fitness value under the maximum number of iterations n.

For the first scheduling sequence L=[l ₀ , l ₁ , l ₂ , ..., l _N-1 ], its fitness function F[L] is:

Among them, P(li) is the total processing time of the product batch, D(li) is the delivery date of the product batch, Begin is the initial time, N is the length of the first scheduling sequence, l ₀ , l ₁ , l ₂ , ..., l _N-1 are the products in the first scheduling sequence.

Under the same other conditions, the larger the fitness value of a product batch sequence, the higher the scheduling efficiency and the more reasonable the optimization.

That is, the step S14 specifically includes:

When the data size does not exceed a preset threshold, two random integers are generated, where the two random integers are different integers between 1 and N, where N is the length of the first scheduling sequence;

Interchange the numerical values of the serial number positions corresponding to the two random integers in the first scheduling sequence to obtain a first comparison sequence;

Calculate and compare the fitness value of the first scheduling sequence sequence with the fitness value of the first comparison sequence sequence; when the fitness value of the first comparison sequence sequence is greater than the fitness value of the first scheduling sequence The first comparison sequence sequence repeats the steps of the value exchange until the maximum number of iterations in the constraint condition is satisfied, and a second sequence sequence sequence is obtained, and the second sequence sequence sequence is the one with the largest adaptation value under the constraint condition. Schedule a sequential sequence.

The step S14 also includes:

When the fitness value of the first comparison sequence is not greater than the fitness value of the first scheduling sequence, repeat the above steps of generating the first comparison sequence until the obtained fitness value of the first comparison sequence is greater than the The fitness value of the first scheduling sequence sequence.

，再分别计 算m个适应值F₁₁，F₁₂，…，F_1m，与

的比例，这样就构造了[0，1]的区间，以m个适应值 F₁₁，F₁₂，…，F_1m，与

的比例将所述[0，1]的区间划分成m个子区间；接着，生成m个0~1 之间的随机数，并将这m个随机数所对应m个子区间的个体挑选出来，形成筛选后种群，例如 某个随机数为0.05，落入在F₁₁的子区间内，此时则选择出F₁₁对应的随机顺序序列L₁₁；以此 类推，可以选出m个随机顺序序列，所述m个随机顺序序列可能存在相同的序列（落入在相同 的子区间内）。 Please refer to FIG. 4 , when the data size does not exceed a preset threshold, the second mathematical optimization model is a genetic algorithm model. At this time, the constraints are the maximum number of iterations n, the number of individuals in the population m, the crossover probability pc and Mutation probability pm, the constraint is the number of times estimated based on experience and target accuracy. After generating m random sequence sequences of length N, that is, the first sequence sequence L ₁₁ , L ₁₂ , ..., L _1m , the individuals of the population are screened by the roulette model. Specifically, the m The fitness values F ₁₁ , F ₁₂ , . . _{. , F 1m of random} sequence sequences L ₁₁ , L _{12 , .} The modified roulette, that is, first calculate the sum of m fitness values F ₁₁ , F ₁₂ , ..., F _1m

, and then calculate m fitness values F ₁₁ , F ₁₂ , . . . , F _1m respectively, and

The proportion of , thus constructing the interval of [0, 1], with m fitness values F ₁₁ , F ₁₂ , . . . , F _1m , and

The ratio of [0, 1] is divided into m sub-intervals; then, m random numbers between 0 and 1 are generated, and the individuals of the m sub-intervals corresponding to the m random numbers are selected to form After screening, the population, for example, a random number of 0.05 falls within the sub-interval of F ₁₁ , at this time, the random sequence sequence L ₁₁ corresponding to F ₁₁ is selected; and so on, m random sequence sequences can be selected, The m random sequence sequences may exist in the same sequence (falling in the same sub-interval).

Next, perform a crossover operation on m random sequence sequences of the screened population, that is, select individuals to be crossed according to the crossover probability pc, and pair them in pairs. For example, the screened population has 100 random sequence sequences, and the crossover probability pc is 0.8, Then select 0.8*100=80 random sequence sequences and match them into 40 pairs of sequence groups; then for each pair of sequence groups, generate a random integer between 1 and N, respectively, for two of the sequence groups. The values of the random sequence sequence from the random integer position are exchanged, and when the numbers are repeated, that is, conflict, they are exchanged at the end conflict, forming m random sequence sequences of the population after the crossover; for example, a sequence group for crossover includes two random sequences. Sequential sequence P ₁ =2,3,1,7,5,6,4,8; P ₂ =3,2,4,8,7,6,1,5; the generated random integer is 3, that is, the The values of the two random sequence sequences P ₁ and P ₂ from the 3rd bit to the left (or to the right) are exchanged in turn, and after the swap, the value 4 of the third bit of P ₁ is repeated with the value 4 of the seventh bit. In this case, it is necessary to exchange the values 4 and 1 of the seventh digit of P ₁ and P ₂ to avoid the problem of repeated conflict of values in the same sequence.

Finally, perform mutation operation on m random sequence sequences of the population after crossover: select individuals to be mutated according to the mutation probability pm, and generate two random integers between 1 and N for each random sequence sequence selected. The subsequences between the corresponding positions of the two random numbers are randomly sorted, and finally the newest population m random sequence sequences L _i1 , L _i2 , ..., _Lim are formed. For example, after crossover, there are 100 random sequence sequences in a population, and the mutation probability pc is 0.05, then 0.05*100=5 random sequence sequences are selected for mutation operation, and for each pair of sequences in these 5 random sequence sequences, generate respectively Two random integers between 1 and N, the sequence is randomly sorted from the subsequence values between the corresponding positions of the two random integers, for example, a sequence is 2, 3, 1, 7, 5, 6, 4 , 8, and the two random integers generated are 3 and 5, then the subsequence values (1, 7, 5) of the 3rd-5th position in the sequence are randomly sorted, for example, the mutated sequence is 2, 3 , 5, 1, 7, 6, 4, 8. In this way, after performing roulette screening, crossover and mutation operations on m random sequence sequences L ₁₁ , L ₁₂ , ..., L _1m , m sequences L' ₁₁ , L' ₁₂ , ..., L' _1m are obtained, and the The fitness values of m sequences L' ₁₁ , L' ₁₂ , ..., L' _1m are obtained, and the optimal sequence F ₁ with the largest fitness value is obtained. In this way, one iteration calculation is completed through the genetic algorithm model.

Next, regenerate new m random sequence sequences, continue to perform roulette screening, crossover, and mutation operations on the new m random sequence sequences to obtain additional m sequences, and then calculate the adaptation of the additional m sequences and obtain the optimal sequence F ₂ with the largest fitness value. In this way, the genetic algorithm model is used to complete multiple iterative calculations until the number of iterations reaches the maximum number of iterations n, then the iteration is stopped, and the fitness values F ₁ , F of n optimal sequences are obtained. _{2 ,} . The above constraints include the maximum number of iterations, the number of individuals in the population, the probability of crossover and the probability of mutation.

The step S14 specifically includes:

When the data size exceeds a preset threshold, perform roulette screening, crossover and mutation operations on the first scheduling sequence under constraints, where the constraints include the maximum number of iterations, the population The number of individuals, the probability of crossover and the probability of mutation;

The second scheduling sequence is determined according to the multiple preferred sequences, and the second scheduled sequence is the preferred sequence with the largest fitness value among the multiple preferred sequences.

The step of performing roulette screening on the first scheduling sequence under constraints specifically includes:

Calculate the fitness values of a plurality of random sequence sequences in the first scheduling sequence sequence respectively, and construct a normalized interval according to the ratio of the fitness value of each random sequence sequence to the sum of the fitness values of all random sequence sequences. The normalized interval includes multiple sub-intervals corresponding to each fitness value, and the value range of the normalized interval is [0, 1];

generating multiple random numbers that are different between 0 and 1, the number of the random numbers is the same as the number of multiple random sequence sequences;

In the normalized interval, the sub-intervals into which each random number falls are determined, and the corresponding random sequence sequence is selected according to the fitness value of the sub-interval that falls into, and a plurality of selected random sequence sequences are obtained.

The step of performing a crossover operation on the first scheduling sequence sequence under constraints specifically includes:

Determine the random sequence sequence to be crossed according to the crossover probability and the multiple random sequence sequences after screening, and pair the random sequence sequences to be crossed in pairs to obtain multiple pairs of sequence groups, each pair of sequence groups includes two random sequence sequences sequence;

Generate a random integer between 1 and N, where N is the length of the first scheduling sequence, and exchange the forward or reverse values of the two random sequence sequences in the sequence group from the random integer position , to obtain multiple random sequence sequences after crossover.

The step of performing mutation operation on the first scheduling sequence sequence under constraints specifically includes:

Determine the random sequence sequence to be mutated according to the mutation probability and multiple random sequence sequences after crossover;

Generate two random integers between 1 and N, where N is the length of the first scheduling sequence, and randomly sort the subsequences between the corresponding positions of the two random integers in the random sequence to be mutated, Obtain multiple mutated random sequence sequences;

A preferred sequence is calculated according to the plurality of mutated random sequence sequences, and the preferred sequence is the sequence with the largest fitness value among the plurality of mutated random sequence sequences.

The step of performing roulette screening, crossover operation and mutation operation on the first scheduling sequence under constraints to obtain multiple preferred sequences further includes:

The steps of generating the first scheduling sequence, roulette screening, crossover operation and mutation operation are repeated to obtain multiple preferred sequences until the maximum number of iterations in the constraints is satisfied.

Step S15: Determine all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traverse each of the state parameters, and determine all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies;

For the specific implementation, please refer to Figure 5 to Figure 8. Aiming at the operation speed, production scale, and scheduling method, a scheduling system based on reinforcement learning is proposed. The decision in any state is used as the scheduling basis to ensure feasible solutions and optimization effects. At the same time, a rapid production scheduling scheme that can be scheduled in real time is formed. In this embodiment, a simulation production scheduling model (that is, a preset simulation production scheduling model) will be established in advance, and environmental parameters of the environment where the simulation production scheduling model is located will be collected, and the collected environmental parameters include equipment information and product information. , process flow and processing time, etc., and define all state parameters, all action parameters and standard returns according to the collected environmental parameters.

In this embodiment, when determining the overall target strategy generated in the reinforcement learning process, each state parameter needs to be traversed first, and all traversal initial strategies corresponding to the traversed state parameters at the current moment are determined in each initial strategy set in advance. Among them, all traversal initial strategies contain traversal state parameters, and the training actions in each traversal initial strategy are different.

It should be noted that, in this embodiment, during the production scheduling process, the product needs to travel in various buffer zones and machines, and the production scheduling is completed after the process flow is completed. For a machine in the production scheduling process, when there are products in the buffer corresponding to the machine, one of the products should be selected for processing. Among them, the buffer zone is a place where products are temporarily placed, before a certain machine or a machine of the same type, and a product processing after machine processing needs to be obtained from the buffer zone. For example, as shown in Figure 4, during the production scheduling process, an idle machine can be used to select a product in the buffer for processing, and each one or more machines corresponds to one or more buffers, such as buffer 1, buffer 2. Buffer 3, etc. Among them, the way of buffer filtering can be shown in Table 1:

Table 1

The way to filter machine types can be shown in Table 2:

Table 2

In this embodiment, the setting of the action parameter can be the number of machine types multiplied by the number of buffers. As shown in Table 1, there are 3 actions in the buffer filtering mode, and there are 3 actions in the filtering machine type mode, then the action parameter The number of can be 9, that is, 9 action parameters.

In this embodiment, the action parameter is based on the machine selecting lot to formulate a one-dimensional action, and sorting the lot of the buffer, specifically: 1. The order of delivery; 2. The customer level; 3. The arrival time of the product from the buffer ; 4. The initial order of the product; and the reward - based on the inverse number of the stage time (unit: second), when there is a lot overdue, subtract the previous positive number (same number × 100) as a penalty.

Step S16: Run the training actions in each of the traversal initial strategies, determine an optimal strategy based on the running results of each of the training actions, determine an overall target strategy according to the optimal strategy corresponding to each of the state parameters, and determine the target overall strategy according to the target The overall strategy is simulated and scheduled, and the third scheduling sequence is obtained;

For the specific implementation, please refer to Figure 5-Figure 8. After all the traversal initial strategies are determined, the simulation scheduling model will execute the training actions in the traversal initial strategy in turn, and when running each training action, the simulation scheduling model will be simulated. The state of the traversal will remain the same as the state parameter of the traversal. After training each training action according to the simulated production scheduling model, the reward value obtained determines the training action with the best reward effect among the training actions, and takes it as the optimal training action, and takes the initial strategy corresponding to the optimal training action as the optimal training action. The optimal strategy includes the traversed state parameters and the optimal training action corresponding to the traversed state parameters. Then it is determined whether the optimal strategy corresponding to all state parameters is obtained. If the optimal strategy corresponding to all state parameters is obtained, it can be determined that the reinforcement learning training has been completed, and the optimal strategy corresponding to all state parameters is used as the target total strategy , and simulate production scheduling according to the target general strategy to obtain a third scheduling sequence.

In this embodiment, the training of the optimal strategy requires the establishment of a Q-value table to store the state S and all actions A that will be taken, ie Q(S, A). For example, as shown in FIG. 8, the Q value table includes action A1, action A2, . . . , action An; state S1, state S2, . . . , state Sn; qn1, . For example, q11=Q(S1, A1), if the qn2 of the state Sn is the largest, it is determined that the best action of the state Sn is A2. At this time, the optimal strategy corresponding to the state Sn includes the state Sn and the action A2.

In this embodiment, by traversing each state parameter, and running all the training actions in the traversal initial strategy corresponding to the traversed state parameter, the optimal strategy is determined, and then the overall target strategy is determined according to the optimal strategy corresponding to each state parameter, Thus, the effectiveness of the obtained target overall strategy is guaranteed.

When it is detected that the simulated production scheduling model has completed the target training action, that is, the target training action has been completed, a corresponding reward will be generated. And in this embodiment, each time the simulated production scheduling model performs an action, a corresponding reward will be generated. Therefore, after the reward corresponding to the target training action is obtained, it can be determined whether the production scheduling learning process based on the traversed state parameters has been completed according to the reward. If not, a new action parameter will be selected as the new action parameter The training action continues to be executed until it is determined that the scheduling learning process based on this traversed state parameter has been completed. And after the target training action is completed, the state of the simulation scheduling model will be converted from the unscheduled state parameter to the traversed state parameter. If it is determined that the production scheduling learning process corresponding to the traversed state parameters has been completed, it is necessary to determine whether the production scheduling learning process corresponding to all the state parameters has been completed. , and determine that the reinforcement learning training is not complete. If the production scheduling learning process corresponding to all the state parameters has been completed, it is determined that the reinforcement learning training has been completed. The method of determining whether the production scheduling learning process corresponding to the traversed state parameters has been completed may be to determine the best action parameters corresponding to the traversed state parameters, that is, determine the rewards corresponding to each action parameter, and select the best effect among the rewards (Compare the reward to be obtained with the standard reward set in advance to determine the reward with the best effect), the reward with the largest reward value, and then use the action parameter corresponding to this reward as the best action parameter, and determine the traversed After the optimal action parameters corresponding to the state parameters are obtained, it can be determined that the production scheduling learning process corresponding to the traversed state parameters has been completed. At this point, the optimal action parameters corresponding to each state parameter in the reinforcement learning training process can be determined, and each group of state parameters and the optimal action parameters corresponding to the group of state parameters can be used as a group of optimal strategies, and then all The optimal strategy corresponding to the state parameters is used as the target total strategy. The return may include one or more of time, delivery, equipment utilization, switching time of machinery and equipment, and recipe switching.

After the target general strategy is obtained, the actual production scheduling operation can be performed according to the target general strategy, and after the production scheduling operation is completed, the production scheduling result of the production scheduling operation will be output. Reinforcement learning training and actual production scheduling operations build a simulated production scheduling model and collect environmental parameters to determine state parameters, action parameters and standard returns, and construct initial strategies based on each state parameter and each action parameter. Then, the learning process of reinforcement learning training is performed on the simulated production scheduling model, that is, initialized data is performed on the state in the simulated production scheduling model, and the state in the simulated production scheduling model is converted into an unscheduled state. Then traverse each state parameter, and determine the initial strategy corresponding to the traversed state parameter, and then obtain the action corresponding to the state (that is, the traversed state parameter) according to the initial strategy, and execute the action to determine whether the production scheduling (ie, production scheduling learning) is completed, If not, in the case of keeping the current traversed state unchanged, acquire a new action and continue to execute it. If so, that is, the production scheduling is completed, it is necessary to determine whether to end the training (that is, the reinforcement learning training). If so, that is, end the training, output the target general strategy, and end the reinforcement learning training process. Then perform the actual production scheduling operation. And in the production scheduling process that starts the production scheduling operation, first obtain all the data, and obtain the actions corresponding to the status according to the target general strategy, execute the actions, and determine whether the production scheduling is completed after the action execution is completed. If not (that is, the scheduling is not completed), continue to obtain new action execution. If yes (that is, the production scheduling is completed), the production scheduling result will be output until the end.

For example, taking a semiconductor processing real-time production scheduling project as an example, if only the processing conditions of the lithography area need to be determined in this production scheduling project, the real environment can be abstracted into an entrance, an exit, a buffer zone and various processing machines. , and design the state (that is, the state parameter) as the number of products waiting to be processed by each machine + the number of products being processed by each machine + the number of buffered products + the number of buffered products being shipped to the machine + the number of imported products + the number of finished processed products at export. The state is represented as a vector or an array, each element of the vector represents a quantity in a specified form, such as the number of products in buffer 1, the number of products in machine 1, ..., the number of products in buffer n, the number of products in machine n, the entry The number of products and the number of completed products. The action (ie action parameter) can be designed as a multi-action of "buffer Lot selection" + "machine selection". The Lot selection can be the highest priority, the first in first out and the least optional machines. The machine selection can be the shortest processing time and the longest machine idle time. The reward can be set to be based on the inverse of the stage time, and when the machine has the same recipe for continuous processing, a positive number is added as a reward. And when determining the best vehicle corresponding to a certain state, it can be determined through a deep regression model, that is, the training action in each of the traversal initial strategies is executed, and the optimal strategy is determined based on the running results of each of the training actions. Steps include:

Build a multilayer perceptron:

Among them, n is the number of layers, which is the number of neurons in each layer, D(S, A) is the function of the state S and the action A as the input, and the output is the Q value;

According to the above multilayer perceptron, the optimal strategy can be obtained, namely:

Among them, D(S, a) is a function of state S and the optimal action a corresponding to state S.

It should be noted that, in the reinforcement learning process in this embodiment, as shown in FIG. 7 , the returns at each stage change dynamically, and it is hoped that the final sum of returns can be maximized through different combinations to further improve other indicators. That is, since there are multiple stages in the production scheduling process, the states and actions corresponding to different stages can be determined, and the rewards corresponding to different stages can be determined, and then the strategies can be summarized and trained according to the states, actions, and rewards. The reward can be stage time (basic) + continuous processing of the same recipe (additional), and the action can be a multi-action of "buffer lot selection" + "machine selection". The Lot selection can be the highest priority, first in first out and optional The minimum number of machines. The machine selection can be the shortest processing time and the longest machine idle time. And the strategy can be obtained by constructing a multi-layer perceptron and using the PPO algorithm for deep reinforcement learning.

Step S17: The production scheduling operation is performed according to the third scheduling sequence.

In the specific implementation, after obtaining the optimally sorted third scheduling sequence, the scheduling operation can be performed according to the third scheduling sequence. Equipment and other resources can help factories rationally allocate capacity, improve resource utilization, reduce production time, balance production lines, and reduce corporate costs.

In addition, please refer to 9, the present invention also provides a semiconductor manufacturing scheduling system for realizing the above-mentioned semiconductor manufacturing scheduling method, the system comprising:

a first determining unit 10, configured to determine a mathematical optimization model according to the data scale of semiconductor manufacturing scheduling;

The first computing unit 20 is configured to generate a first scheduling sequence according to the mathematical optimization model, and obtain a second scheduling sequence by iteratively calculating the first scheduling sequence under constraints by using the mathematical optimization model sequence, the second scheduling sequence sequence is the scheduling sequence sequence with the largest fitness value under the constraint condition, and the constraint condition includes the maximum number of iterations;

The second determining unit 30 is configured to determine the priority of the target attribute of the product to be processed;

The second computing unit 40 is configured to determine all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traverse each of the state parameters, and determine the corresponding state parameters of the traversed state parameters based on each of the initial strategies All traversal initial strategies; and run the training actions in each of the traversed initial strategies, determine the optimal strategy based on the running results of each of the training actions, determine the overall target strategy according to the optimal strategy corresponding to each of the state parameters, and The target general strategy is simulated and scheduled to obtain a third scheduling sequence;

The production scheduling unit 50 is configured to perform a scheduling operation according to the third scheduling sequence.

Wherein, please refer to FIG. 10 , the first computing unit 20 further includes:

The generating subunit 201 is configured to generate two random integers when the data size does not exceed a preset threshold, where the two random integers are different integers between 1 and N, where N is the first scheduling order the length of the sequence;

an exchange subunit 202, configured to exchange the numerical values of the sequence number positions corresponding to two random integers in the first scheduling sequence to obtain a first comparison sequence;

The iterative subunit 203 is configured to calculate and compare the fitness value of the first scheduling sequence with the fitness value of the first comparison sequence; the fitness value of the first comparison sequence is greater than the first scheduling sequence When the fitness value of the sequence is determined, repeat the step of exchanging the numerical values for the first comparison sequence sequence until the maximum number of iterations in the constraint condition is satisfied, and a second scheduling sequence sequence is obtained, and the second scheduling sequence sequence is at: The scheduling sequence sequence with the largest fitness value under the constraints.

In addition, the present invention also provides a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the above-mentioned semiconductor manufacturing scheduling method.

Compared with the prior art, the present invention has the following beneficial effects:

1. By changing the traditional manual production scheduling method, it is easy to face the complex high-mix production environment. The running time is short, and the processing cost for emergencies such as single-insertion failure is small, making the production of enterprises more flexible and efficient.

2. The production plan based on the real production capacity synchronizes the supplier's delivery with the factory scheduling, thereby reducing the inventory cost and transportation cost caused by ordering production raw materials in advance.

3. Through the combination of the global search ability of the intelligent algorithm and the local optimization skills of the reinforcement learning algorithm, the production capacity is effectively and reasonably allocated, the resource utilization rate is improved, and the production cost is reduced.

To sum up, the present invention realizes the rapid target optimization of semiconductor manufacturing scheduling through the combination of the global search ability of the intelligent algorithm and the local optimization skill of the reinforcement learning algorithm, effectively and reasonably allocates the production capacity, improves the resource utilization rate, and reduces the production capacity. cost.

The description and application of the present invention herein is illustrative, and is not intended to limit the scope of the present invention to the above-described embodiments. Variations and variations of the embodiments disclosed herein are possible, and alternative and equivalent various components of the embodiments are known to those of ordinary skill in the art. It should be apparent to those skilled in the art that the present invention may be implemented in other forms, structures, arrangements, proportions, and with other components, materials and components without departing from the spirit or essential characteristics of the invention. Other modifications and changes of the embodiments disclosed herein may be made without departing from the scope and spirit of the invention.

Claims (10)

1. A production scheduling method for semiconductor manufacturing, characterized in that the method comprises the following steps: Obtain the data scale of semiconductor manufacturing scheduling, and determine a mathematical optimization model according to the data scale; A first scheduling sequence is generated according to the mathematical optimization model, and a second scheduling sequence is obtained by iteratively calculating the first scheduling sequence through the mathematical optimization model under constraints. The sequence sequence is a scheduling sequence sequence with the largest fitness value under the constraint condition, and the constraint condition includes the maximum number of iterations; determining all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traversing each of the state parameters, and determining all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies; Run the training actions in each of the traversal initial strategies, determine the optimal strategy based on the running results of each of the training actions, determine the overall target strategy according to the optimal strategy corresponding to each of the state parameters, and perform the strategy according to the overall target strategy. Simulate production scheduling to obtain the third scheduling sequence; The scheduling operation is performed according to the third scheduling sequence. 2 . The method for scheduling semiconductor manufacturing according to claim 1 , wherein the data scale includes the number of products to be processed. 3 . 3 . The semiconductor manufacturing scheduling method according to claim 2 , wherein the second scheduling sequence is obtained by iteratively calculating the first scheduling sequence through the mathematical optimization model under constraints. 4 . The steps specifically include: When the data size does not exceed a preset threshold, two random integers are generated, where the two random integers are different integers between 1 and N, where N is the length of the first scheduling sequence; Interchange the numerical values of the serial number positions corresponding to the two random integers in the first scheduling sequence to obtain a first comparison sequence; Calculate and compare the fitness value of the first scheduling sequence sequence with the fitness value of the first comparison sequence sequence; when the fitness value of the first comparison sequence sequence is greater than the fitness value of the first scheduling sequence The first comparison sequence sequence repeats the steps of the value exchange until the maximum number of iterations in the constraint condition is satisfied, and a second sequence sequence sequence is obtained, and the second sequence sequence sequence is the one with the largest adaptation value under the constraint condition. Schedule a sequential sequence. 4 . The semiconductor manufacturing scheduling method according to claim 2 , wherein the second scheduling sequence is obtained by iteratively calculating the first scheduling sequence through the mathematical optimization model under constraints. 5 . The steps specifically include: When the data size exceeds a preset threshold, perform roulette screening, crossover operation and mutation operation on the first scheduling sequence under constraints, where the constraints include the maximum number of iterations including the maximum number of iterations. The number of iterations, the number of individuals in the population, the probability of crossover and the probability of mutation; The second scheduling sequence is determined according to the multiple preferred sequences, and the second scheduled sequence is the preferred sequence with the largest fitness value among the multiple preferred sequences. 5. The semiconductor manufacturing scheduling method according to claim 4, wherein the step of performing roulette screening on the first scheduling sequence under constraints specifically comprises: Calculate the fitness values of a plurality of random sequence sequences in the first scheduling sequence sequence respectively, and construct a normalized interval according to the ratio of the fitness value of each random sequence sequence to the sum of the fitness values of all random sequence sequences. The normalized interval includes multiple sub-intervals corresponding to each fitness value, and the value range of the normalized interval is [0, 1]; generating multiple random numbers that are different between 0 and 1, the number of the random numbers is the same as the number of multiple random sequence sequences; In the normalized interval, the sub-intervals into which each random number falls are determined, and the corresponding random sequence sequence is selected according to the fitness value of the sub-interval that falls into, and a plurality of selected random sequence sequences are obtained. 6 . The semiconductor manufacturing scheduling method according to claim 5 , wherein the step of performing a cross operation on the first scheduling sequence sequence under a constraint condition specifically comprises: 6 . Determine the random sequence sequence to be crossed according to the crossover probability and the multiple random sequence sequences after screening, and pair the random sequence sequences to be crossed in pairs to obtain multiple pairs of sequence groups, each pair of sequence groups includes two random sequence sequences sequence; Generate a random integer between 1 and N, where N is the length of the first scheduling sequence, and exchange the forward or reverse values of the two random sequence sequences in the sequence group from the random integer position , to obtain multiple random sequence sequences after crossover. 7 . The semiconductor manufacturing scheduling method according to claim 6 , wherein the step of performing a mutation operation on the first scheduling sequence sequence under a constraint condition specifically comprises: 8 . Determine the random sequence sequence to be mutated according to the mutation probability and multiple random sequence sequences after crossover; Generate two random integers between 1 and N, where N is the length of the first scheduling sequence, and randomly sort the subsequences between the corresponding positions of the two random integers in the random sequence to be mutated, Obtain multiple mutated random sequence sequences; A preferred sequence is calculated according to the plurality of mutated random sequence sequences, and the preferred sequence is the sequence with the largest fitness value among the plurality of mutated random sequence sequences. 8. The semiconductor manufacturing scheduling method according to claim 1, wherein the training action includes a delivery sequence, a customer level, an arrival time of products from a buffer zone, and an initial sequence of products. 9. A semiconductor manufacturing scheduling system for realizing the semiconductor manufacturing scheduling method according to any one of claims 1-8, wherein the system comprises: a first determining unit, used for determining a mathematical optimization model according to the data scale of semiconductor manufacturing scheduling; a first computing unit, configured to generate a first scheduling sequence according to the mathematical optimization model, and obtain a second scheduling sequence by iteratively calculating the first scheduling sequence under constraints by the mathematical optimization model , the second scheduling sequence is a scheduling sequence with a maximum fitness value under a constraint condition, and the constraint condition includes a maximum number of iterations; The second determining unit is used to determine the priority of the target attribute of the product to be processed; The second computing unit is configured to determine all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traverse each of the state parameters, and determine the corresponding state parameters of the traversed state parameters based on each of the initial strategies All traversal initial strategies; and run the training actions in each of the traversal initial strategies, determine the optimal strategy based on the running results of each of the training actions, and determine the overall target strategy according to the optimal strategy corresponding to each of the state parameters, and Perform simulated production scheduling according to the target general strategy to obtain a third scheduling sequence; The scheduling unit is used for scheduling operations according to the third scheduling sequence. 10. A computer-readable storage medium on which a computer program is stored, the program being executed by a processor to implement the semiconductor manufacturing scheduling method according to any one of claims 1-8.

Images