Add Filter and Pre Selection Components #65
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There is a temporary measure to deal with custom transformers: there is a wrapping of the code to be imported in both Training and Inference notebooks. - Filter Selection Removes selected features from the dataset. - Pre Selection Removes features with low-variance and high correlation.
fberanizo
requested changes
May 30, 2020
lborro
suggested changes
May 30, 2020
Pre Selection: - Change correlation method - Implementation of the fit method so that we can call the transform method in the Inference file - Import features_after_pipeline Filter: - Changes in contract.json - Implementation of the fit method so that we can call the transform method in the Inference file - Import features_after_pipeline
|
Dá erro quando uso o dataset com categóricos. Ex: titanic |
- Add Wrapping Custom Transformer step in Inference.ipynb - Remove target param from Pre Selection - Handle categorical features in Pre Selection
- Bring back the target variable for Pre Selection
fberanizo
requested changes
Jun 2, 2020
- Change features_to_filter to feature type
- Add target variable in Filter Selection - Refactoring Pre Selection: improving code documentation; remove imports and unused variables; remove target column from features_after_pipeline - Refactoring in attribute engineering components: will no longer need to save numerical_indexes list values using the save_model method, as the model will be able to remeber what are the types of each columns - Change in Inference targets
fberanizo
requested changes
Jun 5, 2020
- Clear Normalizer output
fberanizo
requested changes
Jun 5, 2020
fberanizo
requested changes
Jun 5, 2020
- Add parameter tag in Filter - Get new numerical features indexes after make_column_transformer
fberanizo
requested changes
Jun 6, 2020
|
@lucaslzl @lborro Alguém quer fazer uns teste e trocar os lugares que usam ndarray por DataFrame? Edit: procurei melhor e parece que não é bem como pensei. Ainda é um problema em aberto no scikit-learn: Não adianta fazer a sugestão aí de cima. Testei e não rolou. O jeito parece ser usar um: save_model(
...,
feature_names_in=feature_names_in,
feature_names_out=feature_names_out,
) |
This was referenced Jun 6, 2020
- Save features_after_pipeline and original column
fberanizo
requested changes
Jun 6, 2020
- Make_column_transform remainder change: if the column is not specified, it should not be dropped after transformation
fberanizo
approved these changes
Jun 7, 2020
|
LGTM |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is a temporary measure to deal with custom transformers: there is a wrapping of the code to be imported in both Training and Inference notebooks.
Filter Selection
Removes selected features from the dataset.
Pre Selection
Removes features with low-variance and high correlation.