We've generally focused on enabling cuML with other HPO tools, but we're always looking to better understand user needs. Could you file a Github issue on cuML describing how built-in HPO tooling would affect your work? github.com/rapidsai/cuml/iss…

SVMs are a perfect example of this scenario (results will vary based on data size, CPU, and GPU). In this example, my CPU is relatively weak so I can get benefits even with just 10K rows. gist.github.com/beckernick/f…

Using scikit-learn's GridSearchCV with cuML models should generally give you a large speedup if the model training itself is the bottleneck. If the cuML speedup is > the number of parallel models you can run via n_jobs without sacrificing performance, you'll come out ahead.

Sorry to hear about your experience with PyTorch! As a heads up, you can use cuML estimators with a variety of HPO toolkits including Optuna, Ray/Tune, and most of scikit-learn's model_selection capabilities like GridSearchCV and RandomSearchCV (if you use CPU data)