use "session.runner.run " to predict with tensors，after many times, the memory grow higher and higher then oom #567
Description
Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04 x86_64):Linux x86_64)
- TensorFlow installed from (source or binary):
- TensorFlow version (use command below):
- org.tensorflow tensorflow-core-api 0.4.2 org.tensorflow tensorflow-core-native 1.0.0-rc.2 linux-x86_64
- Java version (i.e., the output of
java -version): java 8 - Java command line flags (e.g., GC parameters):
- Python version (if transferring a model trained in Python):
- Bazel version (if compiling from source):
- GCC/Compiler version (if compiling from source):
- CUDA/cuDNN version:
- GPU model and memory:
Describe the current behavior
we use "session.runner.run " to predict with tensors online，After hours of requests, the instance's memory kept increasing and eventually resulted in an out-of-memory (OOM) (or core dumped )error,
here is my behavior
At the start of the service, we load the session using the loadTensorflowModel method and persist it in memory. During runtime, a complete request process involves: 1. Feeding the Tensor using session.runner.feed, 2. Fetching the results using session.runner.fetch, 3. Obtaining the model's scoring results using the session.runner.run method, 4. Finally, closing both the input tensors and output tensors to complete the request and return the response.
Describe the expected behavior
I desire to keep the memory stable, maintaining it around 65%, rather than continuously increasing over time.
Code to reproduce the issue
package org.example
import org.tensorflow.SavedModelBundle
import org.tensorflow.ndarray.Shape
import org.tensorflow.ndarray.impl.buffer.raw.RawDataBufferFactory
import org.tensorflow.ndarray.buffer.FloatDataBuffer
import org.tensorflow.proto.framework.{ConfigProto, RunOptions}
import org.tensorflow.types.TFloat32
import scala.collection.JavaConversions.iterableAsScalaIterable
import scala.collection.JavaConverters.{asScalaBufferConverter, iterableAsScalaIterableConverter}
import scala.util.Random
object test {
def main(args: Array[String]): Unit = {
valpath = "/xxxxx/tensorflow_test"
val kernel = loadTensorflowModel("/xxxxx/tensorflow_test")
val json = "{\"serving_default_trained_sub_model_score\":[\"NaN\"],\"serving_default_inner_pay_gap_day_seq\":[\"22\",\"95\",\"114\",\"115\",\"122\",\"137\",\"253\"],\"serving_default_input_profile\":[\"6.0\",\"1\",\"6\",\"0.16634308\",\"0.44211066\",\"0.27592358\",\"0.046576172\",\"0.07684306\",\"0.1421662\",\"0\",\"0.622222222222\",\"58\",\"0.513274336283\",\"0.0647766602711\",\"0.0566966144199\",\"0.34084162\",\"0.0806018806897\",\"0.016667\",\"0.071429\",\"3487\",\"1\",\"1\",\"-2.9616807\",\"-1.6372639\",\"-0.7149122\",\"-1.2260734\",\"0.7172737\",\"-0.85318327\",\"-0.9391162\",\"0.093568556\",\"3.7465565\",\"-1.4456742\",\"0.22602487\",\"-1.0320054\",\"1.1451173\",\"0.2524485\",\"-3.0838134\",\"0.588348405415\",\"378\",\"0\",\"0.75\",\"6\",\"0\",\"2.13522742951\",\"6.38635992548\",\"0\",\"2.44948974278\",\"29.1197551762\",\"8600000000\",\"0\",\"1\",\"0\",\"0\",\"0\",\"1\",\"1\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0.118677705526\",\"0.0374854803085\",\"0.229822456837\",\"0.981258451939\",\"0.708709359169\",\"0\",\"0.648167073727\",\"0.0640747249126\",\"0.940500199795\",\"0.18482780456
CA9E
5\",\"0.212323963642\",\"9\",\"3\",\"0\",\"1\",\"0.129026964801\",\"4053.47396349\",\"1708\",\"0.0596125186289\",\"0.0298062593145\",\"0.740323610815\",\"0\",\"0\",\"0\",\"0\",\"0.910581222057\",\"0.421034703002\",\"3\"],\"serving_default_input_product_emb\":[\"1.0954206\",\"0.0007562159\",\"1.4599175\",\"-0.00014710308\",\"-0.0010644994\",\"0.0013906389\",\"-3.69711e-05\",\"-0.001670132\",\"-0.0022175913\",\"0.37936488\",\"0.5284309\",\"0.56634104\",\"1.1236839\",\"0.0017568478\",\"-0.00023513066\",\"0.0010139837\",\"1.1543441\",\"-0.39356884\",\"0.61944026\",\"-5.8332167e-05\"],\"serving_default_input_author_emb\":[\"-3.677482\",\"3.0068984\",\"-2.033637\",\"-3.3155186\",\"-2.1675327\",\"1.2558748\",\"-0.5111591\",\"2.5993257\",\"-0.82787174\",\"-2.3015835\",\"4.0934296\",\"1.0325292\",\"-6.5000277\",\"0.166066\",\"-6.146536\",\"-2.5257614\",\"-0.5164101\",\"3.0859792\",\"-3.6053128\",\"-1.613338\",\"-6.921868\",\"1.3777921\",\"0.15768814\",\"2.6957407\",\"3.7368975\",\"2.6988764\",\"5.4564147\",\"-1.459381\",\"4.033972\",\"-1.4000022\",\"-3.3602858\",\"2.8333883\",\"-5.292586\",\"-0.12055129\",\"-4.8627324\",\"-1.8387526\",\"0.7533025\",\"-3.0386224\",\"8.36378\",\"-2.9174833\",\"-3.233302\",\"-2.0123801\",\"-0.30617282\",\"-0.4304892\",\"6.454887\",\"2.0576859\"],\"serving_default_input_ecom\":[\"0.666666666667\",\"0.666666666667\",\"3.46153846154\",\"0.25\",\"1\",\"0.125\",\"6990\",\"0.110045813064\",\"19980\",\"2.625\",\"0.0967741935484\",\"0.5\",\"12539.5\",\"-1\",\"0\",\"1189.7\",\"-100\",\"43\",\"0\",\"0\",\"0.458333333333\",\"8580.33333333\",\"0.458333333333\",\"3.75\",\"1.63636363636\",\"0.2\",\"9990\",\"0.208333333333\",\"2530.48\",\"1895.29\",\"16900\",\"169\",\"0\",\"0.125\",\"0.1875\",\"0.75\",\"7590\",\"0.63852446987\",\"0.97821619573\",\"0\",\"0.030303030303\",\"0.0330578512397\",\"4.84106818771\",\"4.875\",\"293\",\"0\",\"0\",\"49900\",\"7900\",\"38900\",\"29800\",\"4999\",\"4999\",\"7900\",\"1990\",\"0\",\"0\",\"1990\",\"0.25101561759\",\"0.0606060606061\",\"0.204103887724\",\"54.78\",\"6\",\"8\",\"5\",\"233\",\"109\",\"14\",\"269\",\"0.4\",\"0.657142857143\",\"0.287769784173\",\"0\",\"5\",\"0.775\",\"0.35\",\"32\",\"0.28164556962\",\"0\",\"5\",\"0.359550561798\",\"74\",\"0.831460674157\",\"89\",\"1\",\"0.875\",\"0.439873417722\",\"9.920354\",\"9\",\"1121\",\"9.461538\",\"5\",\"123\",\"3.23\",\"5.67\",\"1.4444444\",\"3.7368073\",\"600\",\"0\",\"0\",\"1\",\"2\",\"1\",\"300\",\"1\",\"300\",\"0.0756034\",\"0\",\"0\",\"0\",\"39900\",\"8991.063\",\"0\",\"2.2173562\",\"0\",\"0.1\",\"0\",\"0.1\",\"13800\",\"19800\",\"1\",\"1990\",\"14990\",\"7584.1665\",\"2\",\"9990\",\"1090\",\"29700\",\"0\",\"7.727525\",\"990\",\"12591.786\",\"2945\",\"-0.9216966\",\"14\",\"0.3887809\",\"-0.034732345\",\"9990\",\"13067535\",\"2679314.5\",\"417107.03\",\"0\",\"0.1113463\",\"0.9142857\",\"2.0270755\",\"1090\",\"0.33333334\",\"0.027777778\",\"0.9166667\",\"3369.009\",\"0.33333334\",\"3065\",\"4490\",\"1\",\"1.1774458\",\"116.37\",\"0.193786031376\",\"4\",\"0.8\",\"116.37\",\"4\",\"0.309203450775\",\"0.352112676056\",\"4\",\"4\",\"149.9\",\"4\",\"0.946666666667\",\"1.70359701196\",\"2.67804173859\",\"28.8862666667\",\"0\",\"2.98689723224\",\"2.97145003934\",\"4.14914227748\",\"0\",\"3\",\"5.74193548387\",\"37\",\"7.3164556962\",\"7.3164556962\",\"44\",\"1.44357080233\",\"0.785714285714\",\"0.272977369149\",\"195198.4\",\"0.161902313592\",\"0.814814814815\",\"0.169973818382\",\"0.822101415924\",\"0.822694820496\",\"1.09660780582\",\"588214.666667\",\"1.36643513432\",\"0.928571428571\",\"1\",\"0.067531948026\",\"0\",\"0.851851851852\",\"0.925925925926\",\"0\",\"0.760891205522\",\"0.394536180641\",\"1.17341440601\",\"1.04531870811\",\"1.03985341166\",\"0.594201949522\",\"0.791215315967\",\"0.857142857143\",\"0.444444444444\",\"0.879436130573\",\"0.15173594712\",\"0.597501416909\",\"0.629928315412\",\"0.687306501548\",\"0.916517323775\",\"0.856118791603\",\"0.31541218638\",\"0.0034784903904\",\"0.737288135593\",\"0.911111111111\",\"0\",\"0.573033707865\",\"0.094674556213\",\"1\",\"72\",\"2\",\"0.0330578512397\",\"3\",\"22.4646332645\",\"38.4798068182\",\"-1.41666666667\",\"-527.033333333\",\"-50.3083333333\",\"51179\",\"238319\",\"0.96\",\"80\",\"239709\",\"0.205882352941\",\"0.0909090909091\",\"0.163742690058\",\"0\",\"0.250975292588\",\"769\",\"0.0469387755102\",\"0.165178571429\",\"2\",\"0\",\"0.186200378072\",\"60\",\"0.0567107750473\",\"0.034703196347\",\"1095\",\"0.4\",\"15104\",\"0.177966101695\",\"18\",\"0.03\",\"1\",\"0.00416666666667\",\"0.0436331255565\",\"2246\",\"0.250396196513\",\"3\",\"0.032967032967\",\"1260\",\"355\",\"0.144543973941\",\"39\",\"1\",\"85\",\"109\",\"0.988235294118\",\"0.0652617642191\",\"0.862595419847\",\"0.486921529175\",\"0\",\"0.049969897652\",\"0.142857142857\",\"2\",\"0.187878787879\",\"0.1231281198\",\"5.9113955\",\"6.3473682\",\"0.36491227\",\"1.4257593\",\"0.4\",\"0\",\"3287785\",\"0.051282052\",\"1489158.6\",\"0.02631579\",\"0.11392166\",\"0.8974359\",\"15153.632\",\"3785.1487\",\"100\",\"3.0971355\",\"0.16866158\",\"0.15\",\"629.5\",\"167.5\",\"0.013411666\",\"0.725\",\"-0.15372944\",\"1.236068\",\"0.075\",\"0.525\",\"0.26585367\",\"1\",\"-1.7106744\",\"0.024163697\",\"0.0051227\",\"0.005735701\",\"0.044593524\",\"0.010041665\",\"0.012656357\",\"0.01228734\",\"0.00412808\",\"0.026462212\",\"-0.113751255\",\"0.020095252\",\"-0.117166795\",\"0.2551187\",\"-0.004284018\",\"-0.0887768\",\"0.083127946\",\"-0.06744456\",\"-0.13762826\",\"0.12308418\",\"0.06441258\",\"-0.09370874\",\"-0.27996\",\"0.20767967\",\"-0.02373123\",\"-0.017548386\",\"-0.17260143\",\"-0.04338026\",\"0.027628133\",\"-0.039716743\",\"0.010191949\",\"-0.1373482\",\"-0.3102747\",\"0.030340211\",\"0.22555368\",\"-0.03646872\",\"2\",\"1\",\"0.0558659217877\",\"0.0111111111111\",\"7\",\"2146\",\"50\",\"0.295857988166\",\"0.530474040632\",\"1\",\"0\",\"0\",\"30\",\"1\",\"60\",\"50\",\"3\",\"95.9909917355\",\"0.153846153846\",\"19\",\"4\",\"1691062874\",\"498\",\"0.338444687842\",\"8\",\"1\",\"48\",\"8\",\"0.923797642332\",\"56\",\"0.225563909774\",\"0.00501253132832\",\"1563725\",\"0.648879510848\",\"0.624060150376\",\"91\",\"471911\",\"0.185463659148\",\"13\",\"0.0325814536341\",\"1000\",\"0.000414957560216\",\"2\",\"1\",\"0.8675\",\"0.599259895701\",\"12150\",\"127917\",\"0.435897435897\",\"0.272686985424\",\"0.418803418803\",\"498784\",\"65900\",\"210339\",\"65900\",\"17\",\"0.145299145299\",\"6000\",\"0.0228310502283\",\"0.230949463186\",\"0.525862068966\",\"18\",\"0.015625\"],\"serving_default_input_type\":[\"1.0\",\"1.0\",\"1.0\",\"1.0\",\"1.0\",\"1.0\",\"1.0\",\"1.0\",\"1.0\",\"1.0\"],\"serving_default_input_aed\":[\"1\",\"1\",\"1\",\"1\",\"1\",\"1\",\"1\",\"2\",\"-1\",\"1\",\"1\",\"-1\",\"1\",\"1\",\"1\",\"305\",\"1\",\"1\",\"NaN\",\"NaN\",\"1\",\"NaN\",\"NaN\",\"1\",\"1\",\"NaN\",\"1\",\"1\",\"2\",\"1\",\"1\",\"1\",\"1\",\"1\",\"1\",\"1\",\"1\",\"1\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"NaN\",\"NaN\",\"-1\",\"1\",\"NaN\",\"1\",\"1\",\"NaN\",\"NaN\",\"11\",\"-1\",\"1\",\"305\",\"NaN\",\"294\",\"11\",\"1\",\"1\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"-1\",\"1\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"294\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"1\",\"1\",\"1.00\",\"NaN\",\"1\",\"1\",\"1\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"NaN\",\"1.00\",\"NaN\",\"NaN\",\"1\",\"-1\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"1\",\"1\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"1\",\"NaN\",\"1\",\"NaN\",\"NaN\",\"0\",\"NaN\",\"1\",\"294\",\"NaN\",\"1\",\"NaN\",\"1\",\"NaN\",\"2\",\"NaN\",\"NaN\",\"1\",\"1\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"0\",\"0\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"0\",\"NaN\",\"-1\",\"NaN\",\"1\",\"NaN\",\"1.00\",\"NaN\",\"1\",\"NaN\",\"1\",\"NaN\",\"NaN\",\"NaN\",\"1\",\"1\",\"1.00\",\"NaN\",\"294\",\"1\",\"NaN\",\"NaN\",\"3\",\"NaN\",\"2.842829193974446\",\"223.3349502429031\",\"4\",\"16.0\",\"11408.0\",\"NaN\",\"NaN\",\"200.0\",\"1621.0\",\"189954.0\",\"0\",\"NaN\",\"5063.5\",\"NaN\",\"3338\",\"6.0\",\"18.0\",\"2.0\",\"0.0\",\"67.2642105263158\",\"NaN\",\"0.3498\",\"0\",\"NaN\",\"1749.0\",\"8\",\"0.3498\",\"18.0\",\"63\",\"NaN\",\"46\",\"0\",\"1762.1875\",\"291408.0\",\"NaN\",\"5000.0\",\"62\",\"1.0\",\"33983\",\"NaN\",\"NaN\",\"1814\",\"3\",\"0.3793103448275862\",\"NaN\",\"1\",\"625.0\",\"5\",\"50.0\",\"20.0\",\"115.2073732718894\",\"NaN\",\"41.0\",\"0\",\"117\",\"34.98\",\"NaN\",\"44\",\"36.48219178082192\",\"33983.0\",\"1749.0\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"33.0\",\"2754.0\",\"0.5141035714285714\",\"450.0\",\"20672.0\",\"4008.0\",\"5000.0\",\"0\",\"NaN\",\"9.0\",\"2\",\"0.0\",\"280000.0\",\"-26\",\"0.7248720048664955\",\"6\",\"0.0\",\"33.0\",\"8\",\"1.0\",\"352.0\",\"4008.0\",\"0.3793103448275862\",\"0.0\",\"3408.0\",\"5000.0\",\"14.0\",\"0.010539393747960032\",\"23.529411764705884\",\"NaN\",\"0.0\",\"1129\",\"8904.09375\",\"33983\",\"0.0\",\"22.0\",\"7\",\"280000.0\",\"1\",\"58.988195615514336\",\"6\",\"66\",\"4.0\",\"0.3498\",\"1500.0\",\"40\",\"7\",\"10.0\",\"0.6206896551724138\",\"28.0\",\"NaN\",\"16167\",\"507.4246752183107\",\"-8498.64705882353\",\"50\",\"18.0\",\"18.0\",\"61.0\",\"34.98\",\"NaN\",\"0.0\",\"2\",\"0.0\",\"0\"],\"serving_default_inner_pay_trade_type_seq\":[\"1\",\"2\",\"1\",\"1\",\"1\",\"1\",\"1\"],\"serving_default_input_douyin\":[\"0.015364182\",\"0.022161229\",\"0.047867857\",\"0.024159236\",\"0.019388612\",\"0.015773335\",\"0.014519561\",\"0.048452497\",\"0.01395968\",\"0.0146678025\",\"0.039789665\",\"0.05345342\",\"0.015731351\",\"0.017113963\",\"0.018419135\",\"0.01637872\",\"0.01319824\",\"0.07278266\",\"0.019222887\",\"0.009909111\",\"0.013782787\",\"0.011220877\",\"0.01734077\",\"0.03857247\",\"0.010123759\",\"1.0840887\",\"0.05621\",\"7\",\"1.496301\",\"2.1177273\",\"1\",\"2.0536497\",\"1.074602\",\"1.109106\",\"3\",\"0\",\"1.6814331\",\"2.0536497\",\"0\",\"1.0679404\",\"1.2790573\",\"1\",\"0.50041\",\"0.030303031\",\"0.03883495\",\"1.5347259\",\"1.7301531\",\"5\",\"1\",\"6\",\"0\",\"0\",\"0\",\"0\",\"1.0794318\",\"6.9139533\",\"22\",\"2576\",\"154\",\"2361\",\"0\",\"0\",\"94\",\"-3\",\"2\",\"7\",\"3\",\"3\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"1\",\"1\",\"0\",\"0\",\"1\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"13943\",\"3\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"7\",\"0\",\"482590\",\"20\",\"1\",\"15766\",\"6\",\"3\",\"33\",\"54\",\"5\",\"5\",\"1\",\"10\",\"2\",\"2\",\"51772\",\"5\",\"1\",\"0\",\"80075\",\"1941945\",\"3\",\"0\",\"4\",\"690\",\"0\",\"0\",\"0\",\"1\",\"2\",\"5\",\"1\",\"6\",\"0\",\"2\",\"4\",\"3889392\",\"171008\",\"11\",\"1\",\"0\",\"0\",\"361567\",\"3\",\"118\",\"1\",\"13\",\"178\",\"6\",\"1\",\"59\",\"4\",\"1\",\"4\",\"69\",\"3\",\"1\",\"2\",\"6\",\"7\",\"399\",\"18\",\"113\",\"2\",\"0\",\"300\",\"77510467\",\"237\",\"2\",\"0\",\"0\",\"14\",\"11779\",\"44\",\"0\",\"0\",\"0\",\"0\",\"17374482\",\"368362000\",\"2\",\"7\",\"11\",\"3\",\"2\",\"4\",\"6\",\"6\",\"0\",\"22\",\"63\",\"18\",\"2963417\",\"132\",\"2\",\"1\",\"1\",\"18\",\"0\",\"4\",\"41\",\"3421\",\"136\",\"0\",\"0\",\"0\",\"1624397\",\"50\",\"24\",\"147\",\"171008\",\"11\",\"0\",\"0\",\"21\",\"2\",\"58\",\"21240\",\"0\",\"70992\",\"4\",\"-0.238098144531\",\"0.334350585938\",\"-0.365112304688\",\"-0.307006835938\",\"0.0566558837891\",\"0.0533294677734\",\"-0.332641601562\",\"0.180725097656\",\"-0.175842285156\",\"-0.195617675781\",\"-0.0202255249023\",\"-0.0471649169922\",\"-0.0341033935547\",\"0.0243453979492\",\"-0.286743164062\",\"0.139465332031\",\"-0.235412597656\",\"-0.475219726562\",\"-0.447631835938\",\"0.145568847656\",\"-0.167419433594\",\"-0.0851745605469\",\"0.0309524536133\",\"-0.197326660156\",\"0.0363006591797\",\"0.0610198974609\",\"-0.212463378906\",\"0.0268936157227\",\"0.155456542969\",\"-0.0501251220703\",\"-0.125793457031\",\"0.317016601562\",\"-0.202087402344\",\"-0.0448455810547\",\"0.143493652344\",\"-0.165100097656\",\"0.0815734863281\",\"0.158508300781\",\"0.196350097656\",\"0.0628967285156\",\"0.404663085938\",\"-0.0847473144531\",\"0.354125976562\",\"-0.195251464844\",\"0.230651855469\",\"-0.115875244141\",\"-0.148254394531\",\"0.25788942\",\"0.0007679774\",\"0.0013244856\",\"0.0018554225\",\"0.0012003026\",\"0.0015898851\",\"0.002294816\",\"0.001212182\",\"0.0016692934\",\"0.0010517475\",\"0.0014232163\",\"0.006211342\",\"0.4402795\",\"1\",\"0\",\"0\",\"0\",\"0.54418945\",\"-9999\",\"0.6550293\",\"0.353232828237\",\"0.189027950168\",\"0.3780559\",\"0.331337230239\",\"0.371525881499\",\"0.872881355932\",\"0\",\"0\",\"0.955555555556\",\"1\",\"60\",\"14\",\"625\",\"0\",\"90\",\"15\",\"16\",\"13\",\"0\",\"394\",\"1\",\"5.22\",\"0.19934468\",\"0.12409516\",\"-1\",\"-1\",\"-1\",\"-1\",\"NaN\",\"1\",\"4\",\"0\",\"1\",\"0\",\"-1\",\"1\",\"1\",\"4455\",\"0\",\"0\",\"1\",\"1.66666666667\",\"1\",\"11\",\"0\",\"6436\",\"58691\",\"0\",\"0\",\"0\",\"0\",\"1.56\",\"2\",\"1\",\"9\",\"1\",\"1\",\"1\",\"1\",\"2\",\"1\",\"31\",\"45\",\"46\",\"47\",\"48\",\"80\",\"16856\",\"9\",\"2\",\"1\",\"932\",\"1\",\"0.0048016105\",\"0.004252207\",\"0.016223682\",\"0.009352278\",\"0.0048474795\",\"0.052074503\",\"0.012187798\",\"6.605\",\"0\",\"0\",\"2.265\",\"0\",\"228.247\",\"35.374\",\"0\",\"446.656\",\"2798.884\",\"0.308499991894\",\"6.346\",\"89.4156996906\",\"95.547\",\"3\",\"0\",\"0\",\"0\",\"0\",\"0\",\"74.831\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0.053314775\",\"1.0900364e-05\",\"1.0758892e-05\",\"1.0781922e-05\",\"1.0713e-05\",\"1.145243e-05\",\"1.08914e-05\",\"1.0883709e-05\",\"1.1084266e-05\",\"0.021303782\",\"1.0953893e-05\",\"1.1278608e-05\",\"1.0947414e-05\",\"1.1745709e-05\",\"1.1423381e-05\",\"1.0942804e-05\",\"1.0833762e-05\",\"0.06448898\",\"1.0900845e-05\",\"1.087283e-05\",\"1.11587615e-05\",\"1.1526876e-05\",\"1.157721e-05\",\"1.077442e-05\",\"0.018841412\",\"1.0981802e-05\",\"1.1307748e-05\",\"1.0594342e-05\",\"1.1384831e-05\",\"1.2230199e-05\",\"1.0779801e-05\",\"1.1218836e-05\",\"0.26357314\",\"1.0976754e-05\",\"0.055758826\",\"0.07733623\",\"1.1496058e-05\",\"0.02944796\",\"1.1229708e-05\",\"1.0900432e-05\",\"1.0779492e-05\",\"0.24316424\",\"1.0952731e-05\",\"0.034155462\",\"1.1045412e-05\",\"1.1203595e-05\",\"1.0781807e-05\",\"1.126557e-05\",\"1.0977601e-05\",\"1.1021473e-05\",\"1.1124115e-05\",\"1.1364056e-05\",\"0.07425524\",\"1.0867204e-05\",\"0.017593198\",\"1.08396425e-05\",\"1.15695875e-05\",\"1.113451e-05\",\"1.0716846e-05\",\"0.043252926\",\"1.1433921e-05\",\"0.0065541463\",\"0.021663623\"],\"serving_default_input_cate\":[\"269.89\",\"0.80441266\",\"968.99\",\"0.5\",\"0.64850247\",\"4\",\"0.875\",\"2\",\"0.083333336\",\"287.9\",\"0.96359307\",\"0.125\",\"0.5\",\"0.64850247\",\"1546.59\",\"0.6111844\",\"17\",\"0.61707264\",\"5\",\"0.8333333\",\"0.510417\",\"4\",\"0.33333334\",\"0.5\",\"0.7916667\",\"-1\",\"0.337542662116\",\"0.30614334471\",\"0.125\",\"0.125\",\"0.115021975529\",\"0.105519471954\",\"0.0487804878049\",\"0.0406504065041\",\"12.1230982203\",\"-0.007178601\",\"-0.0033580181\",\"0.00018617582\",\"-0.021883385\",\"0.016588738\",\"-0.011529422\",\"0.016623275\",\"0.0012452232\",\"-0.020728054\",\"-0.016857037\",\"0.017310059\",\"-0.013332386\",\"-0.0067435307\",\"0.0010332204\",\"0.02989477\",\"-0.026923006\",\"-0.023780176\",\"-0.0146106435\",\"-0.0014884775\",\"-0.029838799\",\"-0.007271569\",\"0.020200195\",\"0.018829055\",\"0.21428572\",\"0.21875\",\"0.33333334\",\"0.27272728\",\"0.39130434\",\"0.3114754\",\"0.2857143\",\"0\",\"0.153846153846\",\"0.11843003413\",\"0.111604095563\",\"0.153846153846\",\"584.3\",\"0.07692308\",\"0.63825566\",\"0.6462925\",\"0.23076923\",\"3015.3\",\"0.17391305\",\"0.81573594\",\"0.87058204\",\"0.3478261\",\"789.65\",\"43\",\"0.22395833\",\"0.25\",\"1110.95\",\"500.4\",\"15\",\"0.171875\",\"0.1875\",\"0.13333334\",\"0.6152951\",\"0.6483995\",\"0.26666668\",\"1259.53\",\"39.95\",\"0.007915123\",\"0.94383126\",\"0\",\"107390\",\"0.81573594\",\"0.87058204\",\"0.15\",\"0.68032366\",\"0.63054687\",\"0.25\",\"7991.68\",\"3555.65\",\"104\",\"0.27272728\",\"0\",\"0\",\"0.617023\",\"2\",\"2\",\"0.25\",\"0.2857143\",\"0.106777206\",\"0.99574804\",\"1\",\"0\",\"0.125\",\"0\",\"0.7114384\",\"0.71580714\"],\"serving_default_inner_pay_pay_status_seq\":[\"1\",\"1\",\"1\",\"1\",\"1\",\"2\",\"1\"],\"serving_default_input_real_time\":[\"0\",\"19\",\"3\",\"0.6838\",\"0\",\"4.6896095\",\"13.2963332251\",\"0.408649257202\",\"0\",\"0\",\"1\",\"327\",\"0\",\"1\",\"0\",\"0\",\"0\",\"19\",\"19\",\"1\",\"1\",\"2203\",\"0\",\"0.09641713\",\"1\",\"3\",\"0.023594523\",\"0.9894\",\"32.18\",\"1\",\"0.033029612\",\"0\",\"1.02078596947\",\"0.1103\",\"0.0523918\",\"0.1985\",\"1\",\"19\",\"1\",\"0.306909126508\",\"0\",\"0\",\"0\",\"1\",\"1\",\"0\",\"1\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"0\",\"32\",\"1\",\"3990\",\"0.0268534399436\",\"0.805897798425\",\"0.999749436232\",\"0.999749436232\",\"0.00399039504911\",\"0.99621077511\",\"23940\",\"21915511\",\"1.0119510009\",\"0.126509543309\",\"5438\",\"0\"],\"serving_default_input_app\":[\"93\",\"11.866646\",\"12.552132\",\"14.91172\",\"7.0242186\",\"-1.96152235015\",\"-1.36175023668\",\"-4.29921700914\",\"0.399202038495\",\"2.01828630522\",\"-1.13376638161\",\"0.999788135215\",\"1.74913257371\",\"-0.810501147957\",\"-2.37487956005\",\"2.16629469634\",\"-0.0908464407634\",\"4.08720124785\",\"-3.4242617182\",\"-3.34396755064\",\"-0.922276056989\",\"0.239808493591\",\"-0.397042896667\",\"-2.17235536906\",\"-4.5696074107\",\"-1.4068919915\",\"-2.09137522737\",\"-13.685564\",\"-12.191576\",\"-11.162037\",\"-13.695179\",\"119.72174761\",\"-399.82718185\",\"-259.414609226\",\"92.980296575\",\"-106.76884925\",\"54.330201587\",\"296.07348201\",\"201.46540676\",\"-207.18532521\",\"263.743178568\",\"-8.448718991\",\"-152.7029836\",\"175.51859297\",\"168.199568\",\"-13.0938081\",\"-85.7716733\",\"108.13419889\",\"-334.64929809\",\"251.194967712\",\"-3.449991865\",\"97.63811605\",\"-424.973489195\",\"-194.497896145\",\"0.083915666\",\"0.21310599\",\"-0.032019783\",\"-0.0027670665\",\"0.6127667\",\"0.54780114\",\"-0.9403887\",\"-0.13972458\",\"-0.23196886\",\"-0.33494487\",\"0.09588502\",\"0.1855776\",\"0.22639273\",\"-0.51628304\",\"-0.114166535\",\"-0.17727913\",\"0.16401836\",\"0.4561239\",\"-0.07739399\",\"-0.0013352555\",\"-0.11681123\",\"0.40487105\",\"-0.07625636\",\"-0.009285554\",\"0.29684278\",\"0.27495435\",\"0.08889192\",\"-0.21101737\",\"-0.09804925\",\"0.08848823\",\"0.35603377\",\"0.09004216\",\"0.017544597\",\"0.13777664\",\"0.19209054\",\"0.25220335\",\"-0.099241585\",\"0.13487557\",\"0.24160497\",\"0.15036109\",\"0.046898365\",\"0.18526399\",\"-0.119316176\",\"0.33652037\",\"-0.03923382\",\"0.048488438\",\"0.36614323\",\"0.23522355\",\"0.1696472\",\"0.07315683\",\"0.12800804\",\"0.7255761\",\"0.04787466\",\"0.26951617\",\"-0.022806883\",\"0.094774276\",\"-0.14585733\",\"0.40569603\",\"0.21212652\",\"-0.037195724\",\"0.102149636\",\"0.02303502\",\"-0.32382822\",\"-0.18476939\",\"-0.23270237\",\"-0.18768175\",\"0.34190777\",\"0.065103315\",\"-0.28565452\",\"-0.2506334\",\"-0.27315792\",\"0.8200944\",\"0.059797514\",\"0.16322254\",\"-0.2014275\",\"-0.16397618\",\"0.14908701\",\"0.042004257\",\"-0.21704954\",\"-0.31267768\",\"0.51314664\",\"-0.33213744\",\"0.31108487\",\"-0.3472758\",\"-0.031644046\",\"0.16807562\",\"0.16617405\",\"0.10363984\",\"0.07000345\",\"0.6260647\",\"0.60179526\",\"-9999\",\"0.59767187\",\"0.172441500884\",\"0.101480108686\",\"0.163144081831\",\"0.013918987\",\"0.004575706\",\"1\"],\"serving_default_input_pay\":[\"0.5\",\"0.1\",\"0\",\"0.1\",\"0\",\"0\",\"0\",\"0.1\",\"2\",\"1\",\"0.16666667\",\"5\",\"0\",\"0\",\"0\",\"0.5\",\"0\",\"0\",\"0\",\"0.5\",\"0.5\",\"0\",\"0\",\"2\",\"2\",\"0.18181819\",\"2\",\"0.45454547\",\"0.16666667\",\"1\",\"1\",\"1\",\"0\",\"0\",\"0\",\"0\",\"1\",\"0.5\",\"2\",\"1\",\"1\",\"715.72\",\"1\",\"1\",\"1754.52\",\"1\",\"1038.8\",\"0.0666666666667\",\"8224.85635359\",\"0.0165745856354\",\"0.0333333333333\",\"0.0295454545455\",\"0.7894737\",\"1\",\"0.44097406\",\"22\",\"0.7946768\",\"0.041430928\",\"934.4\",\"0.010600707\",\"3\",\"2\",\"2\",\"6\",\"8\",\"9\",\"1\",\"0.015675135\",\"44.7\",\"0\",\"4548.7\",\"0\",\"0.032967035\",\"3\",\"0\",\"0\",\"0.010416667\",\"0\",\"0\",\"1\",\"0.005952381\",\"3459.72\",\"3\",\"0.0215827338129\",\"6\",\"6\",\"1\",\"1\",\"1\",\"1\",\"1\",\"0.338444687842\",\"0.0111111111111\",\"0\",\"0.230769230769\",\"4\",\"10.21\",\"2\",\"304.88\",\"7\",\"11.79\",\"1\",\"0.128936170213\",\"0\",\"0.0348837209302\",\"3.99\",\"1\",\"10\",\"1\",\"10\",\"3.35061353862\",\"1.41232920053\",\"3.35061353862\",\"15\",\"0\",\"0.568657627683\",\"0\",\"1\",\"3\",\"0.023320526\",\"0.024752475\",\"0.9528302\",\"0.050552785\",\"0.016806724\",\"0.95652175\",\"0.84615386\",\"0\",\"0\",\"2\",\"0.034998618\",\"0.004201681\",\"8013.87\",\"0.00012056356\",\"2\"],\"serving_default_inner_pay_pay_type_seq\":[\"3\",\"3\",\"5\",\"5\",\"5\",\"5\",\"2\"],\"serving_default_input_graph\":[\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"0.0042347014\",\"NaN\",\"NaN\",\"NaN\",\"NaN\",\"0.0053616506\",\"NaN\",\"NaN\",\"0.016206454\",\"NaN\",\"NaN\"],\"serving_default_inner_pay_create_day_seq\":[\"2\",\"4\",\"4\",\"4\",\"4\",\"4\",\"5\"],\"serving_default_inner_pay_merchant_id_seq\":[\"1\",\"6\",\"1\",\"1\",\"3\",\"1\",\"1\"],\"serving_default_inner_pay_create_hour_seq\":[\"1\",\"4\",\"4\",\"1\",\"4\",\"5\",\"4\"],\"serving_default_input_bdsh\":[\"1.28571428571\",\"0.493068565006\",\"0.439294425001\",\"0.816111885592\",\"0\",\"0\",\"0\",\"0.001403309766532036\",\"5\",\"237260\",\"89270\",\"109840\",\"1\",\"0\",\"1\",\"1300\",\"0\",\"0.272727272727\",\"2\",\"0.881206430502\",\"0.273154075971\",\"0.657142857143\",\"0.98821473\",\"2.2652574\",\"1.4061607\",\"317641\",\"685659\"],\"serving_default_input_address\":[\"0.22043469381\",\"9014.395\",\"1.72085500352\",\"1.93596187896\",\"1.22448979592\",\"117.6\",\"97.4717094017\",\"11404.19\",\"1.26530612245\",\"300\",\"137\",\"1.75625\",\"1.96752533121\",\"-1\",\"0.129032258065\",\"0.548387096774\",\"2\",\"0.0645161290323\",\"0.156626506024\",\"0.156626506024\",\"0.614457831325\",\"0.0602409638554\",\"18\",\"0.148760330579\",\"17\",\"70\",\"0.578512396694\",\"8\",\"0.0661157024793\",\"30\",\"0.147783251232\",\"0.108374384236\",\"0.416666666667\",\"0\",\"0.291666666667\",\"0.0805418641406\",\"0.0839694656489\",\"0\",\"0\",\"0.0667425210323\",\"0\",\"1\",\"1\",\"0\",\"0\",\"6\",\"0.0227272727273\",\"1\",\"0\",\"0.0240749113167\",\"1\",\"1\",\"1\",\"0\",\"0\",\"2\"],\"serving_default_inner_pay_trade_amount_seq\":[\"3.4\",\"67.19\",\"5.9\",\"1.99\",\"69.9\",\"2.8\",\"1.9\"]}"
val data = JacksonUtil.decode[Map[String,Seq[String]]](json)
val runner = kernel.session().runner()
val runTimes = (0 to 1000000).toSeq
runTimes.par.foreach(num =>{
val inputTensorList: java.util.ArrayList[TFloat32] = new java.util.ArrayList[TFloat32]()
//第一步构建输入tensor
data.map{
case (tensorName, featureId) => {
//默认每个inputTensor是一个一维数组
val seq = featureId.map(p => Random.nextFloat()).toSeq
val data: FloatDataBuffer = RawDataBufferFactory.create(seq.toArray,true)
val shape: Shape = Shape.of(1,seq.size)
val tensor = TFloat32.tensorOf(shape,data)
runner.feed(tensorName,tensor)
inputTensorList.add(tensor)
}
}
//构建输出tensor
for(i <- 0 until 2 ){
runner.fetch("StatefulPartitionedCall",i)
}
//开始预测
val output = runner.run().asScala
val scores:Array[Float] = output.map(ten => {
ten.asRawTensor().data().asFloats().getFloat(0)
}).toArray
inputTensorList.asScala.foreach(_.close())
output.foreach(_.close())
print(scores)
})
}
def loadTensorflowModel(path:String):Session= {
val config = ConfigProto.newBuilder
.putDeviceCount("CPU", Runtime.getRuntime.availableProcessors)
.setInterOpParallelismThreads(8) // 设置操作之间的线程数
.setIntraOpParallelismThreads(8) // 设置操作内的线程数
.setOperationTimeoutInMs(500)
.build
val options = RunOptions.newBuilder
.setTimeoutInMs(500)
.build // 设置超时时间为 5 秒
val modelBundle = SavedModelBundle
.loader(s"$path")
.withTags("serve")
.withConfigProto(config)
.withRunOptions(options)
.load
modelBundle.session()
}
}Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
when i stop to request ,then the memory would gradually decrease until it returns to normal levels.
modelFile:after unzip， and load the path
test_model.zip