8000 [BREAKING] gRPC / golang Configuration API refactoring by cmaglie · Pull Request #2565 · arduino/arduino-cli · GitHub
[go: up one dir, main page]

Skip to content

[BREAKING] gRPC / golang Configuration API refactoring #2565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 56 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
8c07823
Moved a batch of function from commands/* subpackage to commands
cmaglie Mar 13, 2024
3d8f50a
Moved a batch of function from commands/* subpackage to commands (par…
cmaglie Mar 13, 2024
3ff8e09
Moved a batch of function from commands/* subpackage to commands (par…
cmaglie Mar 13, 2024
8615985
Moved a batch of function from commands/* subpackage to commands (par…
cmaglie Mar 13, 2024
61e5c46
Moved a batch of function from commands/* subpackage to commands (par…
cmaglie Mar 13, 2024
9e75b4d
Removed unused parameters
cmaglie Mar 13, 2024
f1a015c
Removed unused file
cmaglie Mar 13, 2024
c4ca852
Removed ToRPCStatus in favor of standardized GRPCStatus() interface
cmaglie Mar 13, 2024
23ccd43
Inlining methods in ArduinoCoreServiceImpl (part 1: BoardListAll)
cmaglie Mar 13, 2024
53f03e5
Inlining methods in ArduinoCoreServiceImpl (part 2: BoardDetails)
cmaglie Mar 13, 2024
0df2149
Inlining methods in ArduinoCoreServiceImpl (part 3: BoardList)
cmaglie Mar 13, 2024
f65eff2
Inlining methods in ArduinoCoreServiceImpl (part 4: BoardListWatch)
cmaglie Mar 14, 2024
06d6842
Inlining methods in ArduinoCoreServiceImpl (part 5: Create and Destroy)
cmaglie Mar 14, 2024
8bd0eef
Follow convetion where context must be the first argument
cmaglie Mar 14, 2024
a81069d
Inlining methods in ArduinoCoreServiceImpl (part 6: Init)
cmaglie Mar 14, 2024
917dcc5
Inlining methods in ArduinoCoreServiceImpl (part 7: UpdateIndex)
cmaglie Mar 14, 2024
bb815cf
Inlining methods in ArduinoCoreServiceImpl (part 8: Monitor)
cmaglie Mar 15, 2024
0e434f2
Inlining methods in ArduinoCoreServiceImpl (part 9: Compile)
cmaglie Mar 18, 2024
12cf52e
Inlining methods in ArduinoCoreServiceImpl (part 10: PlatformInstall,…
cmaglie Mar 18, 2024
cfe4b27
Inlining methods in ArduinoCoreServiceImpl (part 11: LibraryDownload,…
cmaglie Mar 18, 2024
a3800fa
Inlining methods in ArduinoCoreServiceImpl (part 12: UpdateLibrariesI…
cmaglie Mar 18, 2024
738ad7c
Inlining methods in ArduinoCoreServiceImpl (part 13: NewSketch, LoadS…
cmaglie Mar 18, 2024
0442319
Inlining methods in ArduinoCoreServiceImpl (part 14: Upload, UploadUs…
cmaglie Mar 18, 2024
65ecbaf
Inlining methods in ArduinoCoreServiceImpl (part 15: SupportedUserFie…
cmaglie Mar 18, 2024
0bdc07a
Inlining methods in ArduinoCoreServiceImpl (part 16: CleanDownloadCac…
cmaglie Mar 25, 2024
3193b7c
Made utility functions private and put them in their own file
cmaglie Mar 18, 2024
c08c59e
Run tests forcing the english language
cmaglie Mar 20, 2024
e8e1574
Use correct env var to force configuration in integration test
cmaglie Apr 19, 2024
9bab3c5
Fixed markdown link
cmaglie Apr 19, 2024
6320f6e
Always use arduino-cli in user-agent string
cmaglie Apr 19, 2024
1fddba7
Removed global *viper.Viper settings instance
cmaglie Mar 20, 2024
4c50868
Implementation of Viper replacement library
cmaglie Apr 17, 2024
0453677
Use custom type to avoid context-values collisions
cmaglie Apr 22, 2024
61b66a7
Fixed test
cmaglie Apr 19, 2024
087a6f5
Fixed FindConfigFileInArgsFallbackOnEnv unit test
cmaglie Apr 19, 2024
9cd9c97
Fixed TestUserAgentHeader and TestProxy
cmaglie Apr 19, 2024
384c30c
Improved error message
cmaglie Apr 21, 2024
6aa742f
Config set adds only unique values
cmaglie Apr 21, 2024
0aafda3
Fixed initialization sequence for 'config init' command
cmaglie Apr 21, 2024
921fa61
Config search now looks into ARDUINO_DIRECTORIES_DATA env too
cmaglie Apr 21, 2024
ad8dbd9
Fixed all config* integration test
cmaglie Apr 21, 2024
4061131
Added missing type schema
cmaglie Apr 21, 2024
fa835de
Added docs
cmaglie Apr 23, 2024
72e05ee
Updated documentation about configuration file format
cmaglie Apr 29, 2024
89a79cb
Fixed a small imperfection in test
cmaglie Apr 30, 2024
240c8b3
Pass context in upload and burn-bootloader
cmaglie Apr 30, 2024
24ac040
Pass down context in all places where it's needed
cmaglie Apr 30, 2024
6cf4eb7
Removed unused parameter
cmaglie Apr 30, 2024
27e9c3e
Tolerate errors in invalid config files, and report them as warnings
cmaglie May 2, 2024
7658db6
Improved handling of incomplete configs array
cmaglie May 2, 2024
a58c350
Disable logging until it is setup in the arduino-cli pre-run
cmaglie May 6, 2024
9585728
Show config-load warning after feedback package initialization
cmaglie May 6, 2024
b279d4b
Fixed 'feedback' package initialization order
cmaglie May 6, 2024
278a331
Applied changes from code review
cmaglie May 6, 2024
c355478
Updated client_example
cmaglie May 6, 2024
cec1fdb
Improved array handling in configmap, and unit-tests
cmaglie May 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Inlining methods in ArduinoCoreServiceImpl (part 12: UpdateLibrariesI…
…ndex)
  • Loading branch information
cmaglie committed May 6, 2024
commit a3800fa2834b64f4a6586b92072f0af1b984d794
62 changes: 47 additions & 15 deletions commands/instances.go
< 8000 tr data-hunk="4738c9bf3f15687315218f83988ecc4d109db5592b886427f0ad6d4d713e249f" class="show-top-border">
Original file line number Diff line number Diff line change
Expand Up @@ -422,47 +422,78 @@ func (s *arduinoCoreServerImpl) Destroy(ctx context.Context, req *rpc.DestroyReq
return &rpc.DestroyResponse{}, nil
}

// UpdateLibrariesIndexStreamResponseToCallbackFunction returns a gRPC stream to be used in UpdateLibrariesIndex that sends
// all responses to the callback function.
func UpdateLibrariesIndexStreamResponseToCallbackFunction(ctx context.Context, downloadCB rpc.DownloadProgressCB) (rpc.ArduinoCoreService_UpdateLibrariesIndexServer, func() *rpc.UpdateLibrariesIndexResponse_Result) {
var result *rpc.UpdateLibrariesIndexResponse_Result
return streamResponseToCallback(ctx, func(r *rpc.UpdateLibrariesIndexResponse) error {
if r.GetDownloadProgress() != nil {
downloadCB(r.GetDownloadProgress())
}
if r.GetResult() != nil {
result = r.GetResult()
}
return nil
}), func() *rpc.UpdateLibrariesIndexResponse_Result {
return result
}
}

// UpdateLibrariesIndex updates the library_index.json
func UpdateLibrariesIndex(ctx context.Context, req *rpc.UpdateLibrariesIndexRequest, downloadCB rpc.DownloadProgressCB) (*rpc.UpdateLibrariesIndexResponse_Result, error) {
logrus.Info("Updating libraries index")
func (s *arduinoCoreServerImpl) UpdateLibrariesIndex(req *rpc.UpdateLibrariesIndexRequest, stream rpc.ArduinoCoreService_UpdateLibrariesIndexServer) error {
syncSend := NewSynchronizedSend(stream.Send)
downloadCB := func(p *rpc.DownloadProgress) {
syncSend.Send(&rpc.UpdateLibrariesIndexResponse{
Message: &rpc.UpdateLibrariesIndexResponse_DownloadProgress{DownloadProgress: p}})
}

pme, release, err := instances.GetPackageManagerExplorer(req.GetInstance())
if err != nil {
return nil, err
return err
}
indexDir := pme.IndexDir
release()

index := globals.LibrariesIndexResource
result := func(status rpc.IndexUpdateReport_Status) *rpc.UpdateLibrariesIndexResponse_Result {
return &rpc.UpdateLibrariesIndexResponse_Result{
LibrariesIndex: &rpc.IndexUpdateReport{
IndexUrl: globals.LibrariesIndexResource.URL.String(),
Status: status,

resultCB := func(status rpc.IndexUpdateReport_Status) {
syncSend.Send(&rpc.UpdateLibrariesIndexResponse{
Message: &rpc.UpdateLibrariesIndexResponse_Result_{
Result: &rpc.UpdateLibrariesIndexResponse_Result{
LibrariesIndex: &rpc.IndexUpdateReport{
IndexUrl: index.URL.String(),
Status: status,
},
},
},
}
})
}

// Create the index directory if it doesn't exist
if err := indexDir.MkdirAll(); err != nil {
return result(rpc.IndexUpdateReport_STATUS_FAILED), &cmderrors.PermissionDeniedError{Message: tr("Could not create index directory"), Cause: err}
resultCB(rpc.IndexUpdateReport_STATUS_FAILED)
return &cmderrors.PermissionDeniedError{Message: tr("Could not create index directory"), Cause: err}
}

// Check if the index file is already up-to-date
indexFileName, _ := index.IndexFileName()
if info, err := indexDir.Join(indexFileName).Stat(); err == nil {
ageSecs := int64(time.Since(info.ModTime()).Seconds())
if ageSecs < req.GetUpdateIfOlderThanSecs() {
return result(rpc.IndexUpdateReport_STATUS_ALREADY_UP_TO_DATE), nil
resultCB(rpc.IndexUpdateReport_STATUS_ALREADY_UP_TO_DATE)
return nil
}
}

// Perform index update
// TODO: pass context
// ctx := stream.Context()
if err := globals.LibrariesIndexResource.Download(indexDir, downloadCB); err != nil {
return nil, err
resultCB(rpc.IndexUpdateReport_STATUS_FAILED)
return err
}

return result(rpc.IndexUpdateReport_STATUS_UPDATED), nil
resultCB(rpc.IndexUpdateReport_STATUS_UPDATED)
return nil
}

// UpdateIndexStreamResponseToCallbackFunction returns a gRPC stream to be used in UpdateIndex that sends
Expand Down Expand Up @@ -593,7 +624,8 @@ func firstUpdate(ctx context.Context, srv rpc.ArduinoCoreServiceServer, instance
// The library_index.json file doesn't exists, that means the CLI is run for the first time
// so we proceed with the first update that downloads the file
req := &rpc.UpdateLibrariesIndexRequest{Instance: instance}
if _, err := UpdateLibrariesIndex(ctx, req, downloadCb); err != nil {
stream, _ := UpdateLibrariesIndexStreamResponseToCallbackFunction(ctx, downloadCb)
if err := srv.UpdateLibrariesIndex(req, stream); err != nil {
return err
}
}
Expand Down
18 changes: 0 additions & 18 deletions commands/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,24 +37,6 @@ type arduinoCoreServerImpl struct {
versionString string
}

// UpdateLibrariesIndex FIXMEDOC
func (s *arduinoCoreServerImpl) UpdateLibrariesIndex(req *rpc.UpdateLibrariesIndexRequest, stream rpc.ArduinoCoreService_UpdateLibrariesIndexServer) error {
syncSend := NewSynchronizedSend(stream.Send)
res, err := UpdateLibrariesIndex(stream.Context(), req,
func(p *rpc.DownloadProgress) {
syncSend.Send(&rpc.UpdateLibrariesIndexResponse{
Message: &rpc.UpdateLibrariesIndexResponse_DownloadProgress{DownloadProgress: p},
})
},
)
if res != nil {
syncSend.Send(&rpc.UpdateLibrariesIndexResponse{
Message: &rpc.UpdateLibrariesIndexResponse_Result_{Result: res},
})
}
return err
}

// Version FIXMEDOC
func (s *arduinoCoreServerImpl) Version(ctx context.Context, req *rpc.VersionRequest) (*rpc.VersionResponse, error) {
return &rpc.VersionResponse{Version: s.versionString}, nil
Expand Down
11 changes: 4 additions & 7 deletions internal/cli/lib/search.go
Original file line number Diff line number Diff line change
Expand Up @@ -107,15 +107,12 @@ func runSearchCommand(srv rpc.ArduinoCoreServiceServer, args []string, namesOnly

logrus.Info("Executing `arduino-cli lib search`")

res, err := commands.UpdateLibrariesIndex(
context.Background(),
&rpc.UpdateLibrariesIndexRequest{Instance: inst, UpdateIfOlderThanSecs: int64(indexUpdateInterval.Seconds())},
feedback.ProgressBar(),
)
if err != nil {
stream, res := commands.UpdateLibrariesIndexStreamResponseToCallbackFunction(ctx, feedback.ProgressBar())
req := &rpc.UpdateLibrariesIndexRequest{Instance: inst, UpdateIfOlderThanSecs: int64(indexUpdateInterval.Seconds())}
if err := srv.UpdateLibrariesIndex(req, stream); err != nil {
feedback.Fatal(tr("Error updating library index: %v", err), feedback.ErrGeneric)
}
if res.GetLibrariesIndex().GetStatus() == rpc.IndexUpdateReport_STATUS_UPDATED {
if res().GetLibrariesIndex().GetStatus() == rpc.IndexUpdateReport_STATUS_UPDATED {
instance.Init(ctx, srv, inst)
}

Expand Down
13 changes: 6 additions & 7 deletions internal/cli/lib/update_index.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,19 +47,18 @@ func runUpdateIndexCommand(srv rpc.ArduinoCoreServiceServer) {
inst := instance.CreateAndInit(ctx, srv)

logrus.Info("Executing `arduino-cli lib update-index`")
resp := UpdateIndex(inst)
resp := UpdateIndex(ctx, srv, inst)
feedback.PrintResult(&libUpdateIndexResult{result.NewUpdateLibrariesIndexResponse_ResultResult(resp)})
}

// UpdateIndex updates the index of libraries.
func UpdateIndex(inst *rpc.Instance) *rpc.UpdateLibrariesIndexResponse_Result {
resp, err := commands.UpdateLibrariesIndex(context.Background(), &rpc.UpdateLibrariesIndexRequest{
Instance: inst,
}, feedback.ProgressBar())
if err != nil {
func UpdateIndex(ctx context.Context, srv rpc.ArduinoCoreServiceServer, inst *rpc.Instance) *rpc.UpdateLibrariesIndexResponse_Result {
req := &rpc.UpdateLibrariesIndexRequest{Instance: inst}
stream, resp := commands.UpdateLibrariesIndexStreamResponseToCallbackFunction(ctx, feedback.ProgressBar())
if err := srv.UpdateLibrariesIndex(req, stream); err != nil {
feedback.Fatal(tr("Error updating library index: %v", err), feedback.ErrGeneric)
}
9E88 return resp
return resp()
}

type libUpdateIndexResult struct {
Expand Down
2 changes: 1 addition & 1 deletion internal/cli/update/update.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ func runUpdateCommand(srv rpc.ArduinoCoreServiceServer, showOutdated bool) {
ctx := context.Background()
inst := instance.CreateAndInit(ctx, srv)

lib.UpdateIndex(inst)
lib.UpdateIndex(ctx, srv, inst)
core.UpdateIndex(ctx, srv, inst)
instance.Init(ctx, srv, inst)
if showOutdated {
Expand Down
0