.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/validation/plot_gridsearch_for_da.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_validation_plot_gridsearch_for_da.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_validation_plot_gridsearch_for_da.py:


Using GridSearchCV with skada
=============================

This example illustrates the use of DA scorer such as :class:`~skada.metrics.ImportanceWeightedScorer`
with `GridSearchCV <https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html>`_.

.. GENERATED FROM PYTHON SOURCE LINES 9-12

We first create a shifted dataset. Then we prepare the pipeline including a
base estimator doing the classification and the DA estimator. We use
:code:`ShuffleSplit` as cross-validation strategy.

.. GENERATED FROM PYTHON SOURCE LINES 12-42

.. code-block:: Python


    import warnings

    import matplotlib.pyplot as plt
    from sklearn.inspection import DecisionBoundaryDisplay
    from sklearn.model_selection import GridSearchCV, ShuffleSplit
    from sklearn.svm import SVC

    from skada import EntropicOTMapping
    from skada.datasets import make_shifted_datasets
    from skada.metrics import PredictionEntropyScorer

    warnings.filterwarnings("ignore")

    RANDOM_SEED = 42
    dataset = make_shifted_datasets(
        n_samples_source=30,
        n_samples_target=20,
        shift="concept_drift",
        label="binary",
        noise=0.4,
        random_state=RANDOM_SEED,
        return_dataset=True,
    )
    X, y, sample_domain = dataset.pack_train(as_sources=["s"], as_targets=["t"])
    X_target, y_target, _ = dataset.pack_test(as_targets=["t"])

    estimator = EntropicOTMapping(base_estimator=SVC(probability=True))
    cv = ShuffleSplit(n_splits=5, test_size=0.3, random_state=RANDOM_SEED)








.. GENERATED FROM PYTHON SOURCE LINES 43-47

We want to perform a grid search to find the best regularization parameter
for the DA estimator. The DA pipeline can directly be used in :code:`GridSearchCV`.
We use the :class:`~skada.metrics.PredictionEntropyScorer`
to evaluate the performance of the DA estimator during the grid search.

.. GENERATED FROM PYTHON SOURCE LINES 47-62

.. code-block:: Python


    reg_e = [0.01, 0.03, 0.05, 0.08, 0.1]

    grid_search = GridSearchCV(
        estimator,
        {"entropicotmappingadapter__reg_e": reg_e},
        cv=cv,
        scoring=PredictionEntropyScorer(),
    )

    grid_search.fit(X, y, sample_domain=sample_domain)

    best_reg_e = grid_search.best_params_["entropicotmappingadapter__reg_e"]
    print(f"Best regularization parameter: {best_reg_e}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Best regularization parameter: 0.08




.. GENERATED FROM PYTHON SOURCE LINES 63-64

Plot the results

.. GENERATED FROM PYTHON SOURCE LINES 64-73

.. code-block:: Python


    plt.plot(
        grid_search.cv_results_["param_entropicotmappingadapter__reg_e"],
        grid_search.cv_results_["mean_test_score"],
    )
    plt.xlabel("Regulariation parameter")
    plt.ylabel("Prediction entropy score")
    plt.show()




.. image-sg:: /auto_examples/validation/images/sphx_glr_plot_gridsearch_for_da_001.png
   :alt: plot gridsearch for da
   :srcset: /auto_examples/validation/images/sphx_glr_plot_gridsearch_for_da_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 74-92

.. code-block:: Python


    DecisionBoundaryDisplay.from_estimator(
        grid_search.best_estimator_,
        X_target,
        alpha=0.8,
        eps=0.5,
        response_method="predict",
    )

    # Plot the target points
    plt.scatter(
        X_target[:, 0],
        X_target[:, 1],
        c=y_target,
        alpha=0.5,
    )
    plt.show()




.. image-sg:: /auto_examples/validation/images/sphx_glr_plot_gridsearch_for_da_002.png
   :alt: plot gridsearch for da
   :srcset: /auto_examples/validation/images/sphx_glr_plot_gridsearch_for_da_002.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 11.356 seconds)


.. _sphx_glr_download_auto_examples_validation_plot_gridsearch_for_da.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_gridsearch_for_da.ipynb <plot_gridsearch_for_da.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_gridsearch_for_da.py <plot_gridsearch_for_da.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_gridsearch_for_da.zip <plot_gridsearch_for_da.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_