hydroPSO - Model-Independent Particle Swarm Optimisation for Environmental Models
RForge.net

hydroPSO

About hydroPSO
GIT access
Download/Files
News
Check results

new

About hydroPSO

What is hydroPSO ?
         hydroPSO is a global optimisation R package implementing a state-of-the-art version of the Particle Swarm Optimisation (PSO) algorithm (SPSO-2011 and SPSO-2007 capable), with a special focus on the calibration of environmental models. 

hydroPSO is parallel-capable, to alleviate the computational burden of complex models.

hydroPSO is model-independent, allowing the user to easily interface any model code with the calibration engine (PSO), and includes a series of controlling options and PSO variants to fine-tune the performance of the optimisation engine. An advanced sensitivity analysis function together with user-friendly plotting summaries facilitate the interpretation and assessment of the calibration results. 


Bugs / comments / questions / collaboration of any kind are very welcomed.

Installation

# Stable version
install.
packages("hydroPSO")

# Latest (under-development) release
install.packages("hydroPSO",, "http://rforge.net/", type="source")

Package Description

Package:hydroPSO
Version:0.3-4-6
Title:Particle Swarm Optimisation, with focus on Environmental Models
Author:Mauricio Zambrano-Bigiarini [aut, cre], Rodrigo Rojas [ctb]
Maintainer:Mauricio Zambrano-Bigiarini
Description:State-of-the-art version of the Particle Swarm Optimisation (PSO) algorithm (SPSO-2011 and SPSO-2007 capable). hydroPSO can be used as a replacement of the 'optim' R function for (global) optimization of non-smooth and non-linear functions. However, the main focus of hydroPSO is the calibration of environmental and other real-world models that need to be executed from the system console. hydroPSO is model-independent, allowing the user to easily interface any computer simulation model with the calibration engine (PSO). hydroPSO communicates with the model through the model's own input and output files, without requiring access to the model's source code. Several PSO variants and controlling options are included to fine-tune the performance of the calibration engine to different calibration problems. An advanced sensitivity analysis function together with user-friendly plotting summaries facilitate the interpretation and assessment of the calibration results. hydroPSO is parallel-capable, to alleviate the computational burden of complex models with "long" execution time. Bugs reports/comments/questions are very welcomed (in English, Spanish or Italian).
Depends:R (>= 2.13.0)
License:GPL (>=2)

Articles 

A model-independent Particle Swarm Optimisation software for model calibration (EMS 2013)
Particle Swarm Optimization for inverse modeling of solute transport in fractured gneiss aquifer (JCH 2014)
Hydrological evaluation of satellite-based rainfall estimates over the Volta and Baro-Akobo Basin (JoH 2013)
Standard Particle Swarm Optimisation 2011 at CEC-2013: A baseline for future PSO improvements (IEEE 2013)
 

Citation

citation("hydroPSO")

Vignette

http://www.rforge.net/hydroPSO/files/hydroPSO_vignette.pdf

IMPORTANT NOTES:
  • The file MF2005.zip, with all the necessary files to run the MODFLOW-2005 examples in the vignette, contains 3 Windows binary filesmf2005.exepreproc.exe and zonbud_hydroPSO.exe. These binary files are distributed in the hope that they will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  In no event shall the authors be liable for any CLAIM, DAMAGES or other LIABILITY, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software. 
  • The file SWAT2005.zip , with all the necessary files to run the SWAT-2005 examples in the vignette, contains 1 Windows binary fileswat2005.exe and  1 UNIX binary fileswat2005.out. Those binary files are distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  In no event shall the authors be liable for any CLAIM, DAMAGES or other LIABILITY, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software. 

Related Material



BugReports

https://www.rforge.net/bugzilla/describecomponents.cgi?product=hydroPSO

Feedback
    
Give us your opinion !


What's New?


NEWS for hydroPSO
--------------------------
0.3-4-6 09-May-2016
        o New field 'Additional_repositories' in the DESCRIPTION file, following the instructions given by CRAN.
        o Minor modification of the `Description' field, following the instructions given by CRAN (avoiding redundancies).
        o CITATION file modified to allow working without package installed (following CRAN request). 
        o DOI field added to CITATION file.
        o CRAN website removed from URL field in DESCRIPTION file (following CRAN request).
        o 'read_out'    : observations (in `obs' or `Observations.txt') are allowed to be of class (Date", "POSIXct", "POSIXt"), not only `Date' as before this enhancement.
        o 'plot_results': observations (obtained with 'read_results.txt') are allowed to be of class (Date", "POSIXct", "POSIXt"), not only `Date' as before this enhancement.
        o 'hydroPSO'    : -) Issue #4. hydroPSO is now able to optimise 1D functions, both R-defined functions or R-external model codes (Thanks to Azza Ahmed and @cjzilverberg!). 
                          -) Issue #3. hydroPSO now is able to correctly apply the 'method="wfips"' when 'topology="random"' (Thanks to @drknexus !). 
                                       For a 2-dimensioanl problem wit 40 particles, it raised an error similar to:
                                         "Error in is.data.frame(x) : 
                                          dims [product 2] do not match the length of object [40]"
                          -) minor bug fixed (Thanks to Aaron Jones !). 
                             This bug might have triggered when the user provided more than 1 parameter set as a first guess for the optimal solution (i.e, a matrix/data.frame), 
                             using the `par' optional argument. Some of the parameters provided by the user might have been incorrectly compared against the `lower' and `upper' values. 


0.3-4   11-Abr-2014
        o Package tested against R version 3.1.0 (2014-04-10) -- "Spring Dance".
        o `lhoat'              : -) when 'normalise=TRUE' the `ParameterSets' output is now given in the real range of each parameter, and not in the 
                                    normalised range [0, 1] as up to version 0.3.2. The 'lhoat-gof.txt' output file has always been written in the real 
                                    range of each parameter when 'normalise=TRUE'. Thanks to Romain Lardy !
                                 -) fixed error: 'Error in Thetas * (UPPER.ini - LOWER.ini) : non-conformable arrays', which was triggered when 
                                    'normalise=TRUE' & 'model.FUN="hydromod"'.
                                 -) now works correctly on Windows machines (when 'parallel="parallelWin"')
                                    (thanks to Dipangkar Kundu !)
                                 -) fixed error : ' Error in argfun(start + i - 1L) : object `model.out.text.file' not found', which was triggered when
                                    'parallel="parallelWin" AND 'write2disk=FALSE'
                                 -) 'parallel_logfile.txt' is now removed before start running this function (when 'parallel!="none"')
                                 -) 'parallel::makeCluster' was replaced by `makeCluster' (and importFrom in NAMESPACE)
                                 -) 'parallel::makeForkCluster' was replaced by `makeForkCluster' (and importFrom in NAMESPACE)
                                 -) 'parallel::detectCores' was replaced by `detectCores' (and importFrom in NAMESPACE)
                                 -) 'parallel::clusterCall' was replaced by `clusterCall' (and importFrom in NAMESPACE)
                                 -) 'parallel::clusterExport' was replaced by `clusterExport' (and importFrom in NAMESPACE)
                                 -) 'parallel::mclapply' was replaced by `mclapply' (and importFrom in NAMESPACE)
                                 -) 'parallel::parRapply' was replaced by `parRapply' (and importFrom in NAMESPACE)
                                 -) 'parallel::clusterApply' was replaced by `clusterApply' (and importFrom in NAMESPACE)
                                 -) 'parallel::stopCluster' was replaced by `stopCluster' (and importFrom in NAMESPACE)
                                 -) `formatC' calls were wrapped into a `suppressWarnings' statement (to avoid unnecessary warnings, due to changes in  R 3.0.2).
        o 'hydroPSO'           : -) The `Dimension' field is now correctly written into the 'PSO_logfile.txt' when 'parallel!=none' (in version 0.3-3 the 
                                    number of cores/nodes was written instead of the dimension of the problem)
                                 -) 'parallel_logfile.txt' is now removed before start running this function (when 'parallel!="none"')
                                 -) now it shows the number of detected cores (when 'parallel!="none" and 'verbose=TRUE')
                                 -) 'parallel:::makeCluster' was replaced by `makeCluster' (and importFrom in NAMESPACE)
                                 -) 'parallel:::makeForkCluster' was replaced by `makeForkCluster' (and importFrom in NAMESPACE)
                                 -) `formatC' calls were wrapped into a `suppressWarnings' statement (to avoid unnecessary warnings, due to changes in  R 3.0.2).
        o `wquantile'          : -) fixed error: 'Error en setTxtProgressBar(pbar, i) : object `pbar' not found', which was triggered when 'verbose=FALSE'.
                                    Thanks to Antonio Piccolboni !
        o 'plot_NparOF'        : `formatC' calls were wrapped into a `suppressWarnings' statement (to avoid unnecessary warnings, due to changes in  R 3.0.2).
        o `verification'       : `formatC' calls were wrapped into a `suppressWarnings' statement (to avoid unnecessary warnings, due to changes in  R 3.0.2).
        o `ModifyInputFile'    : `formatC' calls were wrapped into a `suppressWarnings' statement (to avoid unnecessary warnings, due to changes in  R 3.0.2).
        o 'plot_results'       : 'x11()' was replaced by 'dev.new()'
        o 'plot_particles'     : 'x11()' was replaced by 'dev.new()'
        o `params2ecdf.default': 'x11()' was replaced by 'dev.new()'
        o 'read_out'           : 'require(zoo)' was reaplaced by '!is.na( match("zoo"installed.packages()[,"Package"] ) )'
        o `lhs' package was moved from `Suggests' into `Imports' in NAMESPACE file.
        o `zoo' package removed from `Suggests' (due to changes in CRAN policies) but it stays on `Imports


0.3-3   04-Jun-2013
        o 'hydroPSO'      : fixed error introduced (not intentionally, of course !) in version 0.3-2:
                            Error in FUN(1:number[[1L]], ...) : 
                              unused arguments (Particles = ...)
        o 'pest2hydroPSO' : -) typos fixed in the (automatically created) 'hydroPSO-Rscript.R' file (Thanks to Ramadan Abdel A. !):
                                *) line 38: ending comma removed
                                   out.FUN="read_output"   
                                *) line 57: added ending ', sep=""', to avoid white spaces in the filename
                                   obs.fname <- paste(model.drty, "/PSO.in/", obs.fname, sep="")  
                            -) file `ParamFiles.txt' (automatically created) uses the correct row number for each input file.
                               Up to hydroPSO <= 0.3-2 the row number for each parameter had a shift of 1 line (e.g., it used `5' when the correct 
                               number was 4)
        o NAMESPACE: functions `read.ParameterRanges' and `rLHS' are now exported        

0.3-2   29-May-2013
        o `lhoat'         : -) parallel capable
                            -) much faster than in previous hydroPSO versions, even without using the new `parallel' option (this function was 
                               completely re-written, in order to make it parallel capable)
                            -) new `normalise' parameter for the `control' argument, in order to improve the sampling design when the search space is 
                               not a hypercube
                            -) new `RelativeImportance.Norm' column at the end of the output ranking. The values in `RelativeImportance.Norm' provide
                               normalised values of the relative importance of each parameter. The sum of all the `RelativeImportance.Norm' must be 1. 
                               (Thanks to Rui Esteves for suggesting this feature !)
                            -) `control' argument now allows the following new components: `REPORT', `parallel', `par.nnodes', `par.pkgs', `normalise'
                            -) new argument `...' to allow function arguments that are kept constant during the sensitivity analysis.
        o 'pest2hydroPSO' : -) now it handles correctly cases where the fifth row of `pst.fname' uses tab instead of white spaces as separator 
                               (Thanks to Ramadan Abdel A. !)
        o 'hydroPSO'      : -) minor bug fixed. When the goodness-of-fit value of a given parameter set is not finite, the parameter values are now 
                               correctly written into `BestParamPerIter.txt'
                            -) when 'fn.name!="hydromod"' AND 'normalise=TRUE' (i.e., an R function and NOT an external model run from the system 
                               console) the function `fn' is now evaluated with the correct (non normalised) parameter values 
                               (up to hydroPSO <= 0.3-1-1 `fn' was evaluated with the (non-correct) normalised values when 
                               'fn.name!="hydromod"' AND 'normalise=TRUE')
        
0.3-1-1 10-May-2013
        o Suggested package `multicore' (superseded) was replaced by `parallel'
        o 'hydroPSO: functions from the `multicore' package were replaced by equivalent from the `parallel' package (detectCores, mclapply)

0.3-1   10-May-2013
        o new `SWAT2005.zip' file, with and updated tutorial for calibrating SWAT 2005, compatible with hydroPSO >= 0.3-0   
          The old zip file was renamed to 'SWAT2005_old.zip', which is compatible with hydroPSO <= 0.1-58
          The new `SWAT2005.zip' file now includes a new `swat2005.out' binary file, in order to allow running SWAT2005 in GNU/Linux machines
        o new `MF2005.zip' file, with and updated tutorial for calibrating MODFLOW 2005, compatible with hydroPSO >= 0.3-0   
          The old zip file was renamed to 'MF2005_old.zip', which is compatible with hydroPSO <= 0.1-58    
        o `lhoat'           : -) fixed bug that led to all parameters have the same Ranking (equal to 0) when 'fn!="hydromod"' (Thanks to Rui Esteves !)
                              -) fixed bug that led to the following error message "object `model.out.text.fname' not found" when 'write2disk=FALSE'
                              -) `fn' argument now can be any R function or a character. In the latter case, it can be "hydromod" or the name of a 
                                  valid R function. In previous versions of `lhoat' only a character type was accepted (Thanks to Rui Esteves !)
        o 'hydroPSO'        : -) if a `parallel' directory was not successfully removed during a previous run, it is removed before running the 
                                 algorithm in parallel mode again.
        o 'ReadPlot_results': updated description of the `dp3D.names' argument    
        o 'plot_out'        : `main' argument is not overwritten to "Observed vs `best' Simulation" when this argument is provided and 'ptype=="ts"'
        o 'plot_results'    : `main' argument is now (internally) passed to the 'plot_out' function.
        o 'plot_particles'  : `main' argument is now omitted for the 3D dotty plots ('plot_NparOF' function), in order to avoid confusing (and 
                              sometimes unnecessary) titles when `main' is too long        
        o 'plot_NparOF'     : `main' argument was disabled, in order to avoid confusing titles when 'main is a long text

0.3-0   19-Dec-2012
        o 'hydroPSO'    : -) parallel capable (OS-independent)
                             * new argument `parallel' to define how to parallelise the evaluation of the objective function
                             * new `control' argument `par.nnodes'  to define the number of cores/nodes to be used in a multi-core machine or 
                               network cluster
                             * new `control' argument `par.pkgs' (only needed for Windows machines) with the list of package that need to be 
                               loaded on each core/node 
                          -) Default number of particles for 'method=spso2007' was changed  from 'ceiling(10+2*sqrt(n))' to 'floor(10+2*sqrt(n))', 
                             following Clerc 2012

        o New vignette (14/Dec/2012)
                             
        o 'read_out'    : -) new argument `nsim' used to specify the number of model outputs. It is useful when the model to be calibrated returns NA
                             instead of the simulated values for some particles (e.g., MODFLOW)
                           
        o 'read_results': -) See 'read_out' above

        o 'plot_results': -) See 'read_out' above
                      
                             
##############################################################
# Previous Releases: see old file `ChangeLog' for details #
##############################################################

0.2-0   29-Nov-2012
0.1-58  14-Sep-2012
0.1-57  29-Jun-2012
0.1-56  14-Jun-2012
0.1-55  11-May-2012
0.1-54  01-Apr-2012
0.1.53  23-Mar-2012
0.1.52  09-Mar-2012
0.1.51  24-Feb-2012
0.1.50  23-Feb-2012
0.1.49  13-Feb-2012
0.1.48  22-Jan-2012
0.1.47  16-Jan-2012
0.1.46  29-Nov-2011
0.1-45  11-Nov-2011
0.1-44  28-Oct-2011
0.1-43  17-Oct-2011
0.1-42  08-Oct-2011
0.1-41  05-Sep-2011  
0.1-40  01-Sep-2011  
        24-Jun-2011 
           May-2011
           Apr-2011
        29-Dec-2010 
        23-Dec-2010
           Dec-2010
           Nov-2010
       2009-Oct2010: hydroPSO hibernation :(
          ~Jul-2008: hydroPSO beginning...


See Also

  • hydroGOF: Goodness-of-fit functions for comparison of simulated and observed hydrological time series.
  • hydroTSM: Time series management, analysis and interpolation for hydrological modelling.
  • SWAT2R  : Input/Output functions for the SWAT hydrological model

Visitors