【摘要】 随机优化问题是机器学习与优化交叉领域的一大研究热点,其中经验风险最小化问题尤为突出.提出了一种自适应步长的随机递归梯度算法,结合重要性采样并使用BB方法动态调整步长参数,通过降低随机梯度方差,从而有效地提高了收敛速度.在强凸假设下算法具有线性收敛速度,数值实验表明该算法是有效可行的.更多还原

【Abstract】 Stochastic optimization problems are a major research focus at the intersection of machine learning and optimization, with empirical risk minimization problems being particularly prominent. This paper proposes an adaptive step-size stochastic recursive gradient algorithm, which integrates importance sampling and utilizes the Barzilai-Borwein(BB) method to dynamically adjust the step-size parameter. By reducing the variance of stochastic gradients, the algorithm effectively improves convergence speed. Under the strong convexity assumption, the algorithm achieves a linear convergence rate.Numerical experiments demonstrate that the proposed algorithm is both effective and feasible.更多还原