SIMULTANEOUS PERTURBATION STOCHASTIC APPROXIMATION FOR R

SIMULTANEOUSPERTURBATIONSTOCHASTICAPPROXIMATIONFORREAL-TIMEOPTIMIZATIONOFMODELPREDICTIVECONTROLIrinaBaltchevaFelisaJ.V´azquez-Abad(memberofGERAD)Universit´edeMontr´ealDIRO,CP6128succCentreVille,Montreal,QC,H3C3J7Canadabaltchei,vazquez@ORDSSPSA,CTTheaimofthispaperistosuggestanttrolvariables(temperaturesandinputflowrates)arerealtimecontinuousprocessesandatar-geonicalmodeldiscretisestimeusingasamplinginterval,thustranslatingtheceofthenon-linearityofthecostfunction,commonmethodsforconstrainedoptimizationhavebeenobservtrollersdonotachievetheiroptimalvaluesandthenumericaloptimizationbasedonapproximatinggradi-entsandhessianscannresearchweim-plementamethodologyforglobaloptimizationaddingnoisetotheobservationsofthegradients,UCTIONModelPredictiveControl(MPC)isnowrecognizedintheindustrialworldasaproventechnology,capableofheless,mostoftheindustrialcontreofit,non-linearmodelpredictivecontrol(NMPC)hasreceivedalotofattentioninthelatestyears,bothfromSmarandaCristeaC´esarDePradaUniversidaddeValladolidDISA,PPradodelaMagdalenas/n,Valladolid,47005Spainsmaranda,prada@ointofviewofitsproperties[2]ingtothislastaspect,inearMPCwithconstraintscansolvetheassociatedoptimizationprob-lemeachsamplingtimeusingQPorLPalgorithmsforwhichveryefficientcodesareavailable,NMPCreliesonnon-linearprogramming(NLP)methodssuchasSQP,lschemeshavebeenproposedtodealwiththisproblem,amongthemthewellknownsequentialandsimultaneousapproaches.1Forsequentialsolutions,themodtionandoptimizationcalculationsareperformedse-quentially,rast,simul-taneousmodelsolutionandoptimizationincludesboththemodelstatesandcontrolsasdecisionvariablesandthemodelequngreatlyincreasethesizeoftheop-timizationproblem,cases,computationtimeremainsadiffi-pershowsaglobaloptimizationmethodorientedtoreducethedifficultiesassociatedwiththecomputationofthegradients,inordertofacilitatetheimplementationofNMPCalgorithm,usingthesequentialapproach,appliedtoabenchmarkproblem:ESCRIPTIONTheVanderVussereactionisdescribedindetailin[4]arize,thereisasubstanceininput,’lldenoteby:theconcentrationofproduct(controlled)

Van Der VusseFREACTIONCbQkFigure1:VanderVussetheconcentrationofproduct(measured)thetemperatureinthereactor(measured)thetemperatureinthecoolant(measured)theinputflowofproduct(manipulated)theheatremoval(manipulated)y,ribetheevolutionofthisdynamicalsystem,wein-troducethenon-lineardifferentialequationsrelatedtomassandenergyconservation:vesthefollowingconstraints,whichwillbeincludedintheobjectivefunctionlateron:MODELBASEDPREDICTIVECONTROLLetusdenoteby:thestateattimethecontrolledvariableattimethecontrolattime(manipulatedvari-able)ry(t)TtFigure2:Control:ectiveofthepredictivecontrolistofindthefutureoptimalcontrolsequenceoverafinitehorizontime,whichminimizesthtion,mustassurethatthetrajectoryofissmooth,geofvaluestakenbythemanipulatedvariablectivefunctionisthen:where

ry(t)h2hN_utFigure3:DiscretizedControlappliedtothemanipulatedandcontrolledvariables,canbewrittenas:Afterpenalizingtheconstraintsonthecontrolledvariable,theobjectivefunctionbecomes:whereandarenon-negativeconstants,and,,itishardtocalcu-latethegradient,rtotreatthecontrolconstraints,we’llmake’lltruncateitinthesensethatif,we’llset,andif,we’usapproachestotheoptimizationproblemwerebasedontheSQPalgorithmimplementedintheNAGli-brary,whichusesfiproachhasthedisadvantageofloosingpreci-,REQUIREMENTSToseehowfasttheoptimizationmustbe,letussumma-rizethesimulationbythealgorithmbelow:WHILE(simulationtime)DOWHEN(sampling)atetheoptimalcontrol();hecontrol;e;;wvalueof;;5.;NENDWHILEItisclearthattheoptimalcontrolmustbefoundinlessthanseconds(thesamplingtime)derVussemodelpresentsimportantnon-linearitieswhichmakestheproblemquitediffithers,thisisareasonwhywe’extsection,wepresentthismethod,whichisknownforitseffidreplacesuccessfullythefinitedif-ferencesapproximationsintheVanderVussemoANEOUSPERTURBATIONSTOCHASTICAPPROXIMATION(SPSA)SPSAisadescentmethodcapableoffinfeatureisthegradientapproximationthatre-quiresonlytwomeasurementsoftheobjectivefunction,-callthatwewanttofindtheoptimalcontrol,withlossfunction:BothFiniteDifferencesStochasticApproximation(FDSA)andSPSAusethesameiterativeprocess:whererepresentstheiterate,istheestimateofthegradientoftheobjectivefunction

vector,thecomponentofthesymmetricfinitedifferencegradientestimatoris:FD:RemarkthatFDperturbsonlyonedirectionatthetime,whiletheSPestimatordisturbsalldirectionsinthesametime(thenumeratorisidenticalinallcomponents).ThenumberoflossfunctionmeasurementsneededintheSPSAmethodforeachisalways2,,SPSAusestimesfewerfunctionevaluationsthanFDSA,whichmakesitalotmoreefficient.1052_u0-5-10-10-50510u_1’J(u).dat’g2(x)’control_’g1(x)’control_’Figure4:SPSAvsFDSASimpleexperimenterfollowsapproximatelythesteepestdescentdirection,behav-inglikethegradientmethod(seeFigure4).Ontheotherhand,SPSA,withtherandomsearchdirection,agethough,ittracksitnearlybecausethegradientapproximationisanalmostun-biasedestimatorofthegradient,asshowninthefollowinglemmafoundin[3].thatareallmutuallyindependentwithzero-mean,boundedsecondmoments,edetailedproofisin[3].Themainideaistousecondi-lgebraicmanipulationsimplyingthezeromeanandtheindependenceof,resumesomeofthethehypails,see[5],[3]and[6].Theefficiencyofthemethoddependsontheshapeof,thev,thealgorithmparametersmustsatisfythefollowingconditions:-,whenandgoodchoicewouldbe;a

Figure5:pected,thelatterneededtimeslesscostfunctionevaluations(infact,thedimensionoftheproblemhereis2:).TheresultingcontrolcanbeseeninFigure5,whereareplottedthereferencelevelchangingovertime,theupperandlowerboundsofthecontrolledvariableandtheconcentrationofproductB(),thecon-trolledvariableseemshavingsomedifficultieswhennearitsupperandlowerbounds,adynamicstoppingcriteriaoftheform,whereinmostoftheexperiments,rprecisionwasverycostly,ir,webelievethatabetterchoiceofparameSIONInthisresearchwepresentamethodologyforglobalop-timizationaddingnoisetotheobservshowedthattheadditionofrandomnoisecanmakethecon-trolvariablesattainnearoptimalitymuchfasterthandeter-ministicmethodsasconfition,thismethodcanprovab-ever,findingthesuitedparametersprovedtobechallenging,nces[1],athenascientificedition,1999.[2]-infinitehorimatica,1998.[3]zationofdiscretansactions,29:233–243,1997.[4]eedingsof3rdEuropeanControlConference,pages3247–3252,1995.[5]rand-ceedingsoftheAmericanControlConference,pages756–762,2001.[6]viewofthesimultaneousperturbationmethodforeffipkinsAPLTech-nicalDigest,19(4):482–492,1998.[7]ares,tems:eed-ingsofIMECE2001,NewYork,USA,LEDGEMENTSTheworkofthefirsttwoauthorswassponsoredinpartbyNSERCandFCARGrantsoftheGovernmentofCanadaandQuebec.