【作者】 刘浩；

【导师】 王喆；

【作者基本信息】 吉林大学 ， 计算机技术（专业学位）， 2021， 硕士

【摘要】 云计算环下实验室资源管理平台设计与实现云计算技术的出现与发展为资源管理分配和服务应用部署提供了新的方式,用户可以在网络上依据需求向云平台申请计算、存储等资源,甚至可以申请具体的应用服务。此外,云计算技术也允许用户自定义个性化服务并将其部署在云端,以此节省巨大的硬件、服务实现以及维护成本。本文对容器技术和虚拟机技术进行比较之后,使用容器和编排工具等云计算相关技术,在实验室内部从调度策略、平台设计以及服务部署等三方面搭建了资源管理平台。首先本文研究了Docker、Kubernetes和服务等云相关技术,对其概念、架构与使用进行了简要的说明,其中特别对Kubernetes默认调度器进行了介绍。在此基础上,本文设想两种情况分析默认调度策略存在的问题,并对为不同环境设计高效的调度策略这一课题进行了研究。在高校计算机科研实验室环境中,存在IT基础服务、大学生提交的课程设计和深度学习等任务,这些任务有不同的资源需求、约束条件和生命周期。本文针对实验室的实际情况,设计了调度框架、批处理调度器和动态调度器。调度框架中每种调度器针对特定的任务,批处理调度器负责处理高并发任务,该调度器将收集某一时间段内的所有任务,然后使用多个优化步骤完成统一部署。动态调度器则通过抢占和迁移等策略对争夺稀缺资源的任务进行管理,在动态调度器中本文设计公式对任务的动态优先级进行计算,以该值为基准实现任务的抢占式部署,并通过平台资源的分配情况考虑是否需要任务的迁移。经过与默认调度器的实验对比,本文的调度策略有效提高了平台的资源利用率和系统吞吐率。研究调度策略之后,本文对平台和服务进行了设计实现与部署。首先本文对实验室的硬件、网络等环境进行了配置与规划,随后设计了资源管理平台的整体架构。为了给用户提供深度学习等开发环境,本文设计了基于微服务的软件部署架构,使用容器技术将单机科研软件部署在云端。对于科研实验中复杂的软件环境要求,设计了微服务版本管理应用。此外,考虑到实验室对人员管理、知识继承以及资源共享等方面的特别需求,本文对OAuth2.0、BBS、Nexus和Swift等服务进行实现或二次开发,以完成与实验室个性化需求的适配。本文自建CA签发证书、搭建域名解析系统(DNS)和LDAP服务器,使平台中的所有服务均可以通过自定义域名使用HTTPS协议安全访问,并使用一套账户完成登录。本平台满足研究工作中的大多数典型需求,可以提高科研学习的效率。更多还原

【Abstract】 The development of cloud computing provides new ways to allocate resources and deploy services.Based on this technology,users can apply to the cloud for resources such as computing,storage,and even specific application services through the network.Besides,users can also customize personalized services and deploy them in the cloud.Cloud computing will help users save huge costs of hardware,service implementation and maintenance.After comparing container technology and virtual machine technology,this paper chooses a container technology represented by Docker which is more convenient and lightweight.With cloud computing technologies such as Docker and Kubernetes,we have built a resource management platform inside our laboratory from three aspects:scheduling strategy,platform design and service deployment.First of all,the concepts,architecture and use of cloud computing technologies such as Docker,Kubernetes,and some services are introduced in this paper.Particularly,we introduce the default scheduler of Kubernetes and analyze its defects.Then,how to design effective scheduling strategies for different environments is studied.In the private cloud of computer science labs in universities,there are several kinds of tasks with different resource requirements,constraints,and lifecycles such as IT infrastructure tasks,course design tasks submitted by undergraduate students,and deep learning tasks and so forth.We design the scheduling framework,batch scheduler and dynamic scheduler to handle these tasks.The batch scheduler in the scheduling framework is responsible for batch tasks which will be collected in a period of time and deployed through multiple optimization steps.The dynamic scheduler uses strategies such as preemption and migration to manage tasks competing for scarce resources.The dynamic priority of the task is calculated by the dynamic scheduler to complete the preemptive deployment,and the migration is considered according to the allocation of platform resources.Experiments show that the scheduling strategies proposed in this paper improve resource utilization and system throughput.After studying the scheduling strategy,the platform and services are designed and implemented.Firstly,the hardware and network environment of the laboratory are configured,and then the architecture of the resource management platform is designed.We design a software deployment architecture based on microservices in which a variety of scientific research software is deployed in the form of container.To satisfy complex software environment requirements,a multi version microservice is implemented.In addition,there are special requirements for personnel management,knowledge inheritance,and resource sharing in the laboratory,so this article deploys services such as OAuth2.0,BBS,Nexus,and Swift.Based on self-built CA,domain name system and LDAP server,all services in the platform can be accessed and logged in through domain name and HTTPS protocol.With deploying the platform in our laboratory,it shows that the platform meets most of typical demands in research work and improves research efficiency.更多还原