Source code for "Multi-Purposing Domain Adaptation Discriminators for Pseudo Labeling Confidence" (AdvML at KDD 2019) https://arxiv.org/abs/1907.07802
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Garrett Wilson 138010f1cb display label, option for outputting to files 1 year ago
datasets display label, option for outputting to files 1 year ago
.gitignore local training and evaluation 1 year ago
README.md fixed grammar 1 year ago
checkpoints.py fixed loading checkpoint when no target classifier 1 year ago
file_utils.py track best model based on target classifier 1 year ago
gpu_memory.py fixed GPU memory (copied from time-series-adaptation) 1 year ago
hyperparameter_tuning_commands.py hyperparameter tuning 1 year ago
hyperparameter_tuning_nevergrad.py keep some pending 1 year ago
hyperparameter_tuning_results.py initial hyperparameter tune script 1 year ago
kamiak_config.sh update for Kamiak 1 year ago
kamiak_download.sh tweaks for evaluation 1 year ago
kamiak_eval.srun tweaks 1 year ago
kamiak_hyperparam.srun hyperparameter tuning 1 year ago
kamiak_queue.sh initial Kamiak training scripts 1 year ago
kamiak_tflogs.sh initial Kamiak training scripts 1 year ago
kamiak_train.srun tweaks 1 year ago
kamiak_upload.sh initial Kamiak training scripts 1 year ago
load_datasets.py optionally autotune tf.data 1 year ago
main.py tweaks 1 year ago
main_eval.py tweaks 1 year ago
metrics.py fix metrics and pseudo_label for binary D with logits 1 year ago
models.py for GAN-like training use GRL schedule 1 year ago
pickle_data.py initial hyperparameter tune script 1 year ago
pool.py main_eval.py evaluation script 1 year ago

README.md

Multi-Purposing Domain Adaptation Discriminators for Pseudo Labeling Confidence

Method: instead of using the task classifier's softmax confidence for weighting samples for pseudo labeling, use the discriminator's / domain classifier's confidence based on how source-like the feature representations of the samples appear. In other words, we multi-purpose the discriminator to not only aid in producing domain-invariant representations (like in DANN) but also to provide pseudo labeling confidence.

Steps:

  • Download and preprocess datasets (datasets/generate_tfrecords.py)
  • Optionally view the datasets (datasets/view_datasets.py)
  • Train models (main.py or kamiak_train.srun)
  • Evaluate models (main_eval.py or kamiak_eval.srun)

Training

For example, to train on USPS to MNIST with no adaptation:

./kamiak_queue.sh test1 --model=vada_small --source=usps --target=mnist --method=none

To pseudo label weighting with the domain classifier's confidence (proposed method) or the task classifier's softmax confidence:

./kamiak_queue.sh test1 --model=vada_small --source=usps --target=mnist --method=pseudo
./kamiak_queue.sh test1 --model=vada_small --source=usps --target=mnist --method=pseudo --nouse_domain_confidence --debugnum=1

To instead do instance weighting:

./kamiak_queue.sh test1 --model=vada_small --source=usps --target=mnist --method=instance
./kamiak_queue.sh test1 --model=vada_small --source=usps --target=mnist --method=instance --nouse_domain_confidence --debugnum=1

Or, these but without adversarial training:

./kamiak_queue.sh test2 --model=vada_small --source=usps --target=mnist --method=pseudo --nodomain_invariant
./kamiak_queue.sh test2 --model=vada_small --source=usps --target=mnist --method=pseudo --nouse_domain_confidence --debugnum=1 --nodomain_invariant

Note: you probably need --nocompile_metrics on any SynSigns to GTSRB adaptation, otherwise it may run out of memory. Also, these examples assume you're using SLURM. If not, you can modify kamiak_queue.sh to not queue with sbatch but run with bash.

Evaluating

For example, to evaluate the above "test1" trained models:

sbatch kamiak_eval.srun test1 --eval_batch=2048 --jobs=1