% Iain Murray's "Toolbox" of mostly simple utility functions for Matlab/Octave
% http://homepages.inf.ed.ac.uk/imurray2/code/
%
% Nov 2007 - Started collection
% Jul 2008 - Minor tweaks and documentation cleanup for release
% Nov 2009 - Some additions and tweaks
% Sep 2010 - Some additions and tweaks
% March 2011 - Some additions and tweaks
% June 2011 - Added fpc. Updated links to minimize.m.
%
% Many of these functions I use a lot. Others just seemed fairly generic, so
% were worth maintaining separately from any particular project.
%
% First and foremost, use this command:
% addgenpath - like addpath(genpath(.)), but strips out some
% subdirectories used for version control, testing and so
% on. I use this function to include this "toolbox", which
% may not work properly if you don't!
%
% I have loosely classified the remainder of the commands, but they are a real
% mix of stuff:
%
% Black-box routine helper:
% arg_wrappers - returns functions to transform an argument list into a
% single column vector and back again. Useful for glueing
% together application-specific code and black-box
% optimizers. Used in minimize_args below.
% add_call_counter - wrap a function so that calls to it are counted
%
%
% Gradient-based optimization helpers:
% minimize_args - use Carl Rasmussen's minimize on multiple array variables
% penalize_square - regularize a cost function for use in minimize_args
% utility_to_cost - reverse sign on a function to maximize using minimize_args
% var_checkgrad - Checks partial derivatives wrt a whole argument list
%
% Example usage: maximize utility_fn(arg1, arg2, data) wrt arg1 and arg2, with
% different penalties on the square length of scalar arg1 and length-2 column
% vector arg2. Initial conditions are given by init1 and init2:
% % curry in fixed params
% u_fn = @(arg1, arg2) utility_fn(arg1, arg2, data);
% % Always check gradients (after every code change), or hair loss results!
% err = var_checkgrad(u_fn, 1e-6, init1, init2);
% assert(max(err) < 1e-5);
% % Create a function that should be minimized rather than maximized
% cost_fn = utility_to_cost(u_fn);
% % Create regularized version:
% regularized_cost_fn = penalize_square(cost_fn, {1, [10;100]});
% % And optimize it
% [opt_arg1, opt_arg2, opt_value] = minimize_args(regularized_cost_fn, ...
% -100, init1, init2);
%
% Note that Carl Rasmussen's minimize.m is available from:
% http://www.gaussianprocess.org/gpml/code/matlab/doc/
% Deep link: http://gaussianprocess.org/gpml/code/matlab/util/minimize.m
% (Now comes with less restrictive license, not just research/education.)
% Previous versions required a vector input. Now that Carl's code accepts cells
% and structures, some of the functionality in my wrappers is redundant.
%
%
% Random number generation:
% discreternd - draw samples from a discrete probability distribution
% make_rand_resetter - returns a function to return all RNGs to their current state
%
%
% Indexing:
% ind2subv - return vector positions of linear indexes into an ND-array
% like ind2sub but returns subscripts in rows of a single array
% subv2ind - return linear indexes into an ND-array given by vector positions
% like sub2ind but takes subscripts in rows of a single array
% subv_get - return value in an ND-array given by a vector position
%
%
% Standard processing tasks:
% unique_totals - Given (pos, value) pairs, return unique positions with
% the sum of the values at those positions.
% fpc - Farthest-point clustering
% grabk - First output of fpc: grab K points spread out across data.
% (Included for backwards compatibility. Just call fpc.)
%
%
% Standard functions:
% h2bits - Entropy of a Bernoulli distribution in bits
% h2nats - The same Binary entropy function, but in nats
% hbits - (Cross-)entropy for discrete distributions measured in bits
% hnats - ...and in nats
% kl_bits - KL-divergence between two discrete distributions measured in bits
% kl_nats - ...and in nats
% log1pexp - log(1+exp(x)) avoiding under and overflow (CARE: see help)
% logcumsumexp - Computes log(cumsum(exp(X), dim)) robustly
% logistic - 1./(1+exp(-x))
% logit - log(p./(1-p)), this is the inverse of the logistic
% logplusexp - computes log(exp(A) + exp(B)) robustly, actually log(bsxfun(@plus, exp(A), exp(B)))
% logsumexp - computes log(sum(exp(X), dim)) robustly. (Not quite lightspeed compatible.)
% nchoosekln - like log(nchoosek(N,K)) but without overflow for large K
% nchoosek_owr - like NCHOOSEK but k ordered choices with replacement
% nchoosek_wr - like NCHOOSEK but (unordered) choices with replacement
% plus_diag - add a scalar or vector onto the diagonal of a matrix
% square_dist - compute all NxM square dists between DxN and DxM vectors
%
%
% Functional-style programming utility functions:
% celldeal - returns elements of a cell array as output arguments
% cellmap - maps function over a cell array returning a cell array
% colfun - maps function over cols of an array returning an array
% colmap - maps function over cols of an array returning a cell array
% argdeal - like deal for multiple arguments, but is prepared to throw some away
% rowfun - maps function over rows of an array returning an array
% rowmap - maps function over rows of an array returning a cell array
%
%
% Code readability / neatness:
% DEFAULT - sets a variable to a default value if undefined or empty
% save_except - Save current workspace to disk except variables stated
% PACK_STRUCT - create a structure containing all the named variables
% UNPACK_STRUCT - make all fields of the structure separate variables in the calling workspace
%
%
% Output: diagnostics, plotting, string formatting
% errorbar_str - format a value and error bar as a reasonable-looking string
% ploterrorband - PLOT a line with an error polygon made with FILL
% REAL_FN - mark functions that should only output real numbers (only use while debugging)
% testclose - verbosely check two inputs are close. Use in test cases.
% testequal - verbose version of ISEQUAL for use in test cases.
% testerror - verbosely check that a test function throws an expected error
% testsmall - verbosely check input has small absolute size. Use in test cases.
% teststrequal - verbose version of ISEQUAL special-cased to pairs of strings
%
% System:
% num_cores - attempt to find the number of cores/CPUs on the current machine
%
% Octave only:
%
% Reimplementations of missing Matlab routines (in ./octave directory)
%
% log1p - log(1+x) without underflow
%
% If you have an old version of Octave or Matlab, see:
% http://www.cs.toronto.edu/~murray/code/matlab_octave_missing/
%
% Binary operations readability:
%
% These are a series of functions bsxOP.m that do bsxfun(@OP, x, y)
%
% Having these wrappers makes common operations less cumbersome to read and
% write. They were generated by the python program genbsx.py. They live in a
% sub-directory, as there's lots of them, and they're not very interesting.
% Both R and Python's numpy just define their standard operations to behave
% like these bsx versions. It's unclear why Matlab makes it so cumbersome.
% Still it's better than the old repmat way of doing things.
%
% bsxand
% bsxatan2
% bsxeq
% bsxge
% bsxgt
% bsxhypot
% bsxldivide
% bsxle
% bsxlt
% bsxmax
% bsxmin
% bsxminus
% bsxmod
% bsxne
% bsxor
% bsxplus
% bsxpower
% bsxrdivide
% bsxrem
% bsxtimes
% bsxxor
%
% bsxdiv - This is equivalent to bsxrdivide: z = x./y