gensvm_consistency.h File Reference

Header file for gensvm_consistency.c. More...

#include "gensvm_queue.h"
#include "gensvm_print.h"
#include "gensvm_cv_util.h"
#include "gensvm_cross_validation.h"
#include "gensvm_timer.h"

Include dependency graph for gensvm_consistency.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Functions
struct GenQueue *	gensvm_top_queue (struct GenQueue *q, double percentile)
	Create GenQueue of tasks with performance above a given percentile. More...
int	gensvm_dsort (const void *elem1, const void *elem2)
	Comparison function for doubl. More...
void	gensvm_consistency_repeats (struct GenQueue *q, long repeats, double percentile)
	Run repeats of the GenTask structs in GenQueue to find the best configuration. More...
double	gensvm_percentile (double *values, long N, double p)
	Calculate the percentile of an array of doubles. More...

Detailed Description

Header file for gensvm_consistency.c.

Author

G.J.J. van den Burg

Date

2016-10-24

Copyright

Copyright 2016, G.J.J. van den Burg.

This file is part of GenSVM.

GenSVM is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

GenSVM is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with GenSVM. If not, see http://www.gnu.org/licenses/.

Definition in file gensvm_consistency.h.

Function Documentation

gensvm_consistency_repeats()

void gensvm_consistency_repeats	(	struct GenQueue *	q,
		long	repeats,
		double	percentile
	)

Run repeats of the GenTask structs in GenQueue to find the best configuration.

The best performing tasks in the supplied GenQueue are found by taking those GenTask structs that have a performance greater or equal to the given percentile of the performance of all tasks. These tasks are then gathered in a new GenQueue. For each of the tasks in this new GenQueue the cross validation run is repeated a number of times.

For each of the GenTask configurations that are repeated the mean performance, standard deviation of the performance and the mean computation time are reported.

Finally, the overall best tasks are written to the specified output. These tasks are selected to have both the highest mean performance, as well as the smallest standard deviation in their performance. This is done as follows. First the 99th percentile of task performance and the 1st percentile of standard deviation is calculated. If a task exists for which the mean performance of the repeats and the standard deviation equals these values respectively, this task is found to be the best and is written to the output. If no such task exists, the 98th percentile of performance and the 2nd percentile of standard deviation is considered. This is repeated until an interval is found which contains tasks. If one or more tasks are found, this loop stops.

Parameters

[in]	q	GenQueue of GenTask structs which have already been run and have a GenTask::performance value
[in]	repeats	Number of times to repeat the best configurations for consistency
[in]	percentile	percentile of performance to determine which tasks to repeat

Definition at line 128 of file gensvm_consistency.c.

Here is the call graph for this function:

gensvm_dsort()

int gensvm_dsort	(	const void *	elem1,
		const void *	elem2
	)

Comparison function for doubl.

Parameters

[in]	elem1	number 1
[in]	elem2	number 2

Returns

comparison of number 1 larger than number 2

Definition at line 275 of file gensvm_consistency.c.

gensvm_percentile()

double gensvm_percentile	(	double *	values,
		long	N,
		double	p
	)

Calculate the percentile of an array of doubles.

The percentile of performance is used to find the top performing configurations. Since no standard definition of the percentile exists, we use the method used in MATLAB and Octave. Since calculating the percentile requires a sorted list of the values, a local copy is made first.

Parameters

[in]	values	array of doubles
[in]	N	length of the array
[in]	p	percentile to calculate ( 0 <= p <= 100.0 ).

Returns

the p-th percentile of the values

Definition at line 296 of file gensvm_consistency.c.

Here is the call graph for this function:

gensvm_top_queue()

struct GenQueue* gensvm_top_queue	(	struct GenQueue *	q,
		double	percentile
	)

Create GenQueue of tasks with performance above a given percentile.

This function constructs a GenQueue of the GenTask instances in the provided GenQueue which have a performance at or above the given percentile of the performance of all elements in the provided GenQueue. This can be used to determine which hyperparameter configurations belong to the top x-% of all tasks in terms of performance.

See also

gensvm_consistency_repeats(), gensvm_percentile()

Note

This function assumes that for each task in the given GenQueue, the GenTask::perf element has been set.

Parameters

[in]	q	a complete GenQueue struct
[in]	percentile	the desired percentile

Returns

a GenQueue struct with GenTasks which are at or above the desired percentile of performance

Definition at line 59 of file gensvm_consistency.c.

Here is the call graph for this function: