8000 Adding synthetic data generation example. · dotnet/TorchSharpExamples@6ee17c8 · GitHub
[go: up one dir, main page]
Skip to content

Commit 6ee17c8

Browse files
NiklasGustafssonNiklasGustafsson
committed
Adding synthetic data generation example.
1 parent 1218eaa commit 6ee17c8

File tree

2 files changed

+391
-0
lines changed

2 files changed

+391
-0
lines changed

‎.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -352,3 +352,5 @@ MigrationBackup/
352352
# Ionide (cross platform F# VS Code tools) working folder
353353
.ionide/
354354

355+
*.dat.x
356+
*.dat.y

‎tutorials/CSharp/synthetic_data.ipynb

Lines changed: 389 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,389 @@
1+
{
2+
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"metadata": {},
7+
"source": [
8+
"# Using TorchSharp to Generate Synthetic Data for a Regression Problem\n",
9+
"\n",
10+
"This tutorial is based on a [PyTorch example](https://jamesmccaffrey.wordpress.com/2023/06/09/using-pytorch-to-generate-synthetic-data-for-a-regression-problem/) posted by James D. McCaffrey on his blog, ported to TorchSharp.\n",
11+
"\n",
12+
"Note that we're taking some shortcuts in this example -- rather than writing the data set as a text file that can be loaded from any modeling framework, we're saving the data as serialized TorchSharp tensors. Is should be straight-forward to modify the tutorial to write the data sets as text, instead."
13+
]
14+
},
15+
{
16+
"cell_type": "code",
17+
"execution_count": null,
18+
"metadata": {
19+
"dotnet_interactive": {
20+
"language": "csharp"
21+
},
22+
"polyglot_notebook": {
23+
"kernelName": "csharp"
24+
},
25+
"vscode": {
26+
"languageId": "polyglot-notebook"
27+
}
28+
},
29+
"outputs": [],
30+
"source": [
31+
"#r \"nuget: TorchSharp-cpu\"\n",
32+
"\n",
33+
"using TorchSharp;\n",
34+
"using static TorchSharp.TensorExtensionMethods;"
35+
]
36+
},
37+
{
38+
"attachments": {},
39+
"cell_type": "markdown",
40+
"metadata": {},
41+
"source": [
42+
"Neural networks can be used to generate data as well as train. The synthetic data can then be used to evaluate different models to see how well they can copy the behavior of the network used to produce the data.\n",
43+
"\n",
44+
"First, we will create the model that will be used to generate the synthetic data. Later, we'll construct a second model that will be trained on the data the first model generates."
45+
]
46+
},
47+
{
48+
"cell_type": "code",
49+
"execution_count": null,
50+
"metadata": {
51+
"vscode": {
52+
"languageId": "polyglot-notebook"
53+
}
54+
},
55+
"outputs": [],
56+
"source": [
57+
"class Net : torch.nn.Module<torch.Tensor,torch.Tensor>\n",
58+
"{\n",
59+
" private torch.nn.Module<torch.Tensor,torch.Tensor> hid1;\n",
60+
" private torch.nn.Module<torch.Tensor,torch.Tensor> oupt;\n",
61+
"\n",
62+
" public Net(int n_in) : base(nameof(Net))\n",
63+
" {\n",
64+
" var h = torch.nn.Linear(n_in, 10);\n",
65+
" var o = torch.nn.Linear(10,1);\n",
66+
"\n",
67+
" var lim = 0.80;\n",
68+
" torch.nn.init.uniform_(h.weight, -lim, lim);\n",
69+
" torch.nn.init.uniform_(h.bias, -lim, lim);\n",
70+
" torch.nn.init.uniform_(o.weight, -lim, lim);\n",
71+
" torch.nn.init.uniform_(o.bias, -lim, lim);\n",
72+
"\n",
73+
" hid1 = h;\n",
74+
" oupt = o;\n",
75+
"\n",
76+
" RegisterComponents();\n",
77+
" }\n",
78+
" public override torch.Tensor forward(torch.Tensor input)\n",
79+
" {\n",
80+
" using var _ = torch.NewDisposeScope();\n",
81+
" var z = torch.tanh(hid1.call(input));\n",
82+
" z = torch.sigmoid(oupt.call(z));\n",
83+
" return z.MoveToOuterDisposeScope();\n",
84+
" }\n",
85+
"}"
86+
]
87+
},
88+
{
89+
"attachments": {},
90+
"cell_type": "markdown",
91+
"metadata": {},
92+
"source": [
93+
"Now that we have our generative network, we can define the method to create the data set. If you compare this with the PyTorch code, you will notice that we're relying on TorchSharp to generate a whole batch of data at once, rather than looping. We're also using TorchSharp instead of Numpy for the noise-generation."
94+
]
95+
},
96+
{
97+
"cell_type": "code",
98+
"execution_count": null,
99+
"metadata": {
100+
"dotnet_interactive": {
101+
"language": "csharp"
102+
},
103+
"polyglot_notebook": {
104+
"kernelName": "csharp"
105+
},
106+
"vscode": {
107+
"languageId": "polyglot-notebook"
108+
}
109+
},
110+
"outputs": [],
111+
"source": [
112+
"void CreateDataFile(Net net, int n_in, string fileName, int n_items)\n",
113+
"{\n",
114+
"\n",
115+
" var x_lo = -1.0;\n",
116+
" var x_hi = 1.0;\n",
117+
"\n",
118+
" var X = (x_hi - x_lo) * torch.rand(new long[] {n_items, n_in}) + x_lo;\n",
119+
"\n",
120+
" torch.Tensor y;\n",
121+
"\n",
122+
" using (torch.no_grad()) {\n",
123+
" y = net.call(X);\n",
124+
" }\n",
125+
"\n",
126+
" // Add some noise in order to not make it too easy to train...\n",
127+
" y += torch.randn(y.shape) * 0.01;\n",
128+
"\n",
129+
" // Make sure that the output isn't negative.\n",
130+
" y = torch.where(y < 0.0, 0.01 * torch.randn(y.shape) + 0.01, y);\n",
131+
"\n",
132+
" // Save the data in two separate, binary files.\n",
133+
" X.save(fileName + \".x\");\n",
134+
" y.save(fileName + \".y\");\n",
135+
"}"
136+
]
137+
},
138+
{
139+
"cell_type": "code",
140+
"execution_count": null,
141+
"metadata": {
142+
"dotnet_interactive": {
143+
"language": "csharp"
144+
},
145+
"polyglot_notebook": {
146+
"kernelName": "csharp"
147+
},
148+
"vscode": {
149+
"languageId": "polyglot-notebook"
150+
}
151+
},
152+
"outputs": [],
153+
"source": [
154+
"var net = new Net(6);"
155+
]
156+
},
157+
{
158+
"attachments": {},
159+
"cell_type": "markdown",
160+
"metadata": {},
161+
"source": [
162+
"Create the data files."
163+
]
164+
},
165+
{
166+
"cell_type": "code",
167+
"execution_count": null,
168+
"metadata": {
169+
"dotnet_interactive": {
170+
"language": "csharp"
171+
},
172+
"polyglot_notebook": {
173+
"kernelName": "csharp"
174+
},
175+
"vscode": {
176+
"languageId": "polyglot-notebook"
177+
}
178+
},
179+
"outputs": [],
180+
"source": [
181+
"CreateDataFile(net, 6, \"train.dat\", 200);\n",
182+
"CreateDataFile(net, 6, \"test.dat\", 40);"
183+
]
184+
},
185+
{
186+
"attachments": {},
187+
"cell_type": "markdown",
188+
"metadata": {},
189+
"source": [
190+
"Load them again. This is just to demonstrate how to get the data from disk."
191+
]
192+
},
193+
{
194+
"cell_type": "code",
195+
"execution_count": null,
196+
"metadata": {
197+
"dotnet_interactive": {
198+
"language": "csharp"
199+
},
200+
"polyglot_notebook": {
201+
"kernelName": "csharp"
202+
},
203+
"vscode": {
204+
"languageId": "polyglot-notebook"
205+
}
206+
},
207+
"outputs": [],
208+
"source": [
209+
"var X_train = torch.Tensor.load(\"train.dat.x\");\n",
210+
"var y_train = torch.Tensor.load(\"train.dat.y\");\n",
211+
"var X_test = torch.Tensor.load(\"test.dat.x\");\n",
212+
"var y_test = torch.Tensor.load(\"test.dat.y\");"
213+
]
214+
},
215+
{
216+
"attachments": {},
217+
"cell_type" F438 ;: "markdown",
218+
"metadata": {},
219+
"source": [
220+
"Create another class, with slightly different logic, and train it on the generated data set."
221+
]
222+
},
223+
{
224+
"cell_type": "code",
225+
"execution_count": null,
226+
"metadata": {
227+
"dotnet_interactive": {
228+
"language": "csharp"
229+
},
230+
"polyglot_notebook": {
231+
"kernelName": "csharp"
232+
},
233+
"vscode": {
234+
"languageId": "polyglot-notebook"
235+
}
236+
},
237+
"outputs": [],
238+
"source": [
239+
"class Net2 : torch.nn.Module<torch.Tensor,torch.Tensor>\n",
240+
"{\n",
241+
" private torch.nn.Module<torch.Tensor,torch.Tensor> hid1;\n",
242+
" private torch.nn.Module<torch.Tensor,torch.Tensor> oupt;\n",
243+
"\n",
244+
" public Net2(int n_in) : base(nameof(Net2))\n",
245+
" {\n",
246+
" hid1 = torch.nn.Linear(n_in, 5);\n",
247+
" oupt = torch.nn.Linear(5,1);\n",
248+
"\n",
249+
" RegisterComponents();\n",
250+
" }\n",
251+
" public override torch.Tensor forward(torch.Tensor input)\n",
252+
" {\n",
253+
" using var _ = torch.NewDisposeScope();\n",
254+
" var z = torch.nn.functional.relu(hid1.call(input));\n",
255+
" z = torch.sigmoid(oupt.call(z));\n",
256+
" return z.MoveToOuterDisposeScope();\n",
257+
" }\n",
258+
"}"
259+
]
260+
},
261+
{
262+
"attachments": {},
263+
"cell_type": "markdown",
264+
"metadata": {},
265+
"source": [
266+
"We create an instance of the second network, choose a loss to use, and then we're ready to train it."
267+
]
268+
},
269+
{
270+
"cell_type": "code",
271+
"execution_count": null,
272+
"metadata": {
273+
"dotnet_interactive": {
274+
"language": "csharp"
275+
},
276+
"polyglot_notebook": {
277+
"kernelName": "csharp"
278+
},
279+
"vscode": {
280+
"languageId": "polyglot-notebook"
281+
}
282+
},
283+
"outputs": [],
284+
"source": [
285+
"var model = new Net2(6);\n",
286+
"\n",
287+
"var loss = torch.nn.MSELoss();"
288+
]
289+
},
290+
{
291+
"attachments": {},
292+
"cell_type": "markdown",
293+
"metadata": {},
294+
"source": [
295+
"A standard training loop. It ends with evaluating the trained model on the training set."
296+
]
297+
},
298+
{
299+
"cell_type": "code",
300+
"execution_count": null,
301+
"metadata": {
302+
"dotnet_interactive": {
303+
"language": "csharp"
304+
},
305+
"polyglot_notebook": {
306+
"kernelName": "csharp"
307+
},
308+
"vscode": {
309+
"languageId": "polyglot-notebook"
310+
}
311+
},
312+
"outputs": [],
313+
"source": [
314+
"var learning_rate = 0.01f;\n",
315+
"\n",
316+
"Console.WriteLine(\" initial loss = \" + loss.forward(model.forward(X_train), y_train).item<float>().ToString());\n",
317+
"\n",
318+
"var optimizer = torch.optim.SGD(model.parameters(), learning_rate);\n",
319+
"\n",
320+
"for (int i = 0; i < 10000; i++) {\n",
321+
" // Compute the loss\n",
322+
" using var output = loss.forward(model.forward(X_train), y_train);\n",
323+
"\n",
324+
" // Clear the gradients before doing the back-propagation\n",
325+
" model.zero_grad();\n",
326+
"\n",
327+
" // Do back-progatation, which computes all the gradients.\n",
328+
" output.backward();\n",
329+
"\n",
330+
" optimizer.step();\n",
331+
"}\n",
332+
"\n",
333+
"Console.WriteLine(\" final loss = \" + loss.forward(model.forward(X_train), y_train).item<float>());"
334+
]
335+
},
336+
{
337+
"attachments": {},
338+
"cell_type": "markdown",
339+
"metadata": {},
340+
"source": [
341+
"The thing we're really curious about is how the second model does on the test set, which it didn't see during training. If the loss is significantly greater than the one from the training set, we need to train more, i.e. start another epoch. If the test set loss doesn't get closer to the training set loss with more epochs, we may need more data."
342+
]
343+
},
344+
{
345+
"cell_type": "code",
346+
"execution_count": null,
347+
"metadata": {
348+
"dotnet_interactive": {
349+
"language": "csharp"
350+
},
351+
"polyglot_notebook": {
352+
"kernelName": "csharp"
353+
},
354+
"vscode": {
355+
"languageId": "polyglot-notebook"
356+
}
357+
},
358+
"outputs": [],
359+
"source": [
360+
"loss.forward(model.forward(X_test), y_test).item<float>()"
361+
]
362+
},
363+
{
364+
"cell_type": "code",
365+
"execution_count": null,
366+
"metadata": {
367+
"dotnet_interactive": {
368+
"language": "csharp"
369+
},
370+
"polyglot_notebook": {
371+
"kernelName": "csharp"
372+
},
373+
"vscode": {
374+
"languageId": "polyglot-notebook"
375+
}
376+
},
377+
"outputs": [],
378+
"source": []
379+
}
380+
],
381+
"metadata": {
382+
"language_info": {
383+
"name": "python"
384+
},
385+
"orig_nbformat": 4
386+
},
387+
"nbformat": 4,
388+
"nbformat_minor": 2
389+
}

0 commit comments

Comments
 (0)
0