《论文（设计）-基于并行文件的分层Web 服务器群.doc》由会员分享，可在线阅读，更多相关《论文（设计）-基于并行文件的分层Web 服务器群.doc（6页珍藏版）》请在三一文库上搜索。
1、专业好文档基于并行文件的分层Web服务器群王燮1 王 鹏2,1 向 令1 黄松林1 (1.成都信息工程学院并行计算实验室 四川,成都 610225; 2. 电子科技大学 四川 成都 610054）摘 要：构建了并行文件Web服务器机群。该机群采用分层结构，每层节点均作为调度器(最后层除外)和数据服务器。各层节点对外提供数据服务，并根据预设的计量标尺判断数据服务是否过量，并将过量工作转移至下层节点。建立了基于ajax和vml技术的监控服务器，对所有节点性能进行实时监控。关 键 词：分层Web服务器；计量标尺；并行文件I/O；MPI 中图法分类号: TP393 文献标识码: A随着Web2.0的兴
2、起和传统Web的不断发展，各企业以及个人对Web服务器的性能要求越来越高。目前Web服务器大都采用昂贵的专用服务器，如浪潮NL380D。现有的Web服务器机群中，只是对数据服务进行了简单的平衡。这两种传统的Web服务，其Web请求调度带来的额外开销成了它们的瓶颈，限制其规模的扩大。为了实现低成本下为客户提供高效的Web服务这一目标，本文提出基于并行文件和分层拓扑的Web服务器模型，利用多层结构对繁重的Web请求/响应分而治之。通过对各节点的负载均衡，大幅提高了Web服务的性能。同时对所有节点进行实时监控，实现对各节点更有效的管理。1并行相关技术及并行文件I/O高性能计算技术大多采用并行技术，现
3、有的并行技术中以MPI最为流行。MPI(Message Passing Interface)是目前最重要的一个基于消息传递并行编程工具，它具有移植性好，功能强大，效率高等许多优点，而且有多种不同的免费，高效，实用的实现版本，几乎所有的并行计算机厂商都提供对它的支持，成为了事实上的并行编程标准。MPI-2提供并行文件I/O ，能够方便的实现文件的并行读写。基于这些原因，本文选用了MPI-2作为服务端系统的开发平台。2. 基于并行文件分层式Web服务器模型2.1Web服务器拓扑结构图1. 基于并行文件的多层Web服务器负载均衡模型【注：1，接收用户请求。2，分析消息中包含的文档大小，并行文件I/O
4、。3，溢出数据传至下层节点。4，响应客户请求。】服务器节点采用分层结构。如图1所示，系统由多层分布式的服务器组成，每层由多个节点构成(除根节点外)，上层节点是下层所有节点的父节点,下层节点有多个父节点,每层节点数目根据实际需要决定。层数和各层节点数均可扩展，增加层内节点数目或服务深度即可提升系统处理大批量请求的能力。由于调度服务器可能在下层搜寻负载最轻的调度节点和进行并行I/O操作，为降低该节点的工作强度，同层节点数目不宜过多。当然，其规模应以实际需求和单个服务器性能而定。2.2 分层Web服务器的调度策略所有节点既是数据服务器，除末层外又作为调度服务器。凡调度服务器均预设有计量标尺，用于判断
5、当前节点的数据服务是否过量。节点的调度或数据服务均在本地进行，过量数据部分则传递至下层作相应处理。根节点至末层节点, 计量标尺值则呈递增趋势（同层间各标尺值应相同），使得各节点总的工作量趋于一致，负载更加均衡。为实现服务器在繁忙状态也能及时响应所有请求的这一目标，我们采用了计量标尺式调度策略。每次调度时调度服务器需要完成四项工作：分析用户请求所要求返回用户浏览器的所有文档的大小，并将数据返回给用户；计算当前溢出的数据文档大小相对下层服务器节点数的平均值；以中得到的平均值为标准对溢出服务队列的文档作并行I/O处理，并采用不连续数据发送方式进行消息传递；调度转移，同时准备接收父节点新发来的消息。以
6、上四步中，步必须在服务队列有溢出才执行。服务器与浏览器之间约定文档的类型需在其头部进行描述，在进行文档并行I/O时，各子文档头部同样需要做类似标记，便于事后数据的整合。通过调度服务器基于内容发起的调度，所有子层节点协同响应，且各节点需要处理的累计文档大小相同。建议服务队列的存储采取小文档优先原则，这样既能减轻消息传递的通讯量，还有利于可能出现的文件的递归式并行I/O。2.3 分层Web服务器的MPI实现本文提出的分层式Web服务器模型是建立在并行文件I/O和MPI的消息传递机制基础之上的。并行文件I/O是并行计算中最复杂的一种并行计算方式，最基本的并行I/O操作，包括指定显式偏移的并行文件读写
7、，使用独立文件指针的并行文件读写和共享文件读写三种主要类型。MPI-2提出了一套并行I/O的标准接口，明确了对文件的各种并行操作和管理方法，所有子节点的进程可以用其提供的方法来对单一文件进行访问。例如MPI_FILE_OPEN是一个组调用（即通信域中的所有进程都必须按给定的文件名来执行它，各个进程使用的文件名要相同，给定的打开方式也要相同，但是各个进程传递给正在运行时的信息可以互不相同），该调用返回一个文件句柄，以后各个进程通过文件句柄就能具体操作文件6。目前最重要的并行编程模型是消息传递和数据并行，MPI的消息传递是目前使用最广泛的并行编程方法之一，已被成功应用于高性能数值计算领域。由于发送
8、操作和接收操作在处理器对之间进行，被称为点到点通信操作，能高效的完成各调度服务器节点的消息传递工作。数据并行即将相同的操作同时作用于不同的数据，数据并行编程模型提供给编程者一个全局的地址空间，这种形式的语言本身就提供并行执行的语义，对于编程者来说，只需要简单地指明执行什么样的并行操作和并行操作的对象，就实现了数据并行的编程6。2.4性能分析采用分层Web均衡策略,系统能动态的根据各服务节点实际负载进行分层调度，能最大程度的提高资源利用率，同时实现较高的服务器性能。通过数据的本地服务有效减少了昂贵的消息传递操作，避免了通讯造成的等待时间；而数据的并行服务又大大缩短了相对单机服务器返回大文件所耗用
9、的更长的服务时间；基于并行文件I/O操作，实现了对任意文档，无论类型、大小均可平均分配至子网各数据服务器节点上。层间异步有序、协调工作，层内协同完成服务，部分资源出现故障不会影响到系统的其他服务。合理设定标尺能有效控制每台节点的具体工作量，实现了在大量请求下Web服务器负载的自动均衡而且使系统具有良好的扩展性。3. 机群监控系统该模型中设有一个监控服务器，如图1所示。该监控服务器采用了B/S模式，利用ajax异步请求和vml技术(画各种状态图)对所有节点进行实时监控。其访问权限可设只对内网，也可设外网访问。所有节点将自己被监控的数据发送至监控服务器，监控服务器将数据存入数据库中保存，同时将数据
10、发送至客户端，使系统管理员能对系统进行实时监控。监控服务器根据管理员的请求，与指定节点建立SOCKET连接，使监控服务器能够控制节点运行各种命令(如kill节点的进程)，然后根据管理员提交的命令对节点进行管理。即实现了一个基于B/S终端。4. 总结本文讨论了调度和数据服务协同化，提出了一种采用分层Web服务器模型，该模型采用分层结构及并行文件读取基础可以达到很高的性价比，实现了低成本、高性能服务器部署方法。理论上只需合理设定各调度节点计量标尺值、调整并行粒度大小和控制系统规模，即可适应高强度的数据服务，为解决大数据量Web服务器的负载均衡问题提供了一种新的思路。本文作者创新点: 1.提出了分层
11、结构Web服务器架构。2. 首次采用并行文件I/O读写和基于消息传递的并行编程函数库MPI完成数据传递。3.提出了计量标尺的概念。参考文献：1 Zhang XL,Barrientos M,Chen JBSeltzer M.HACC : An architecture for cluster-based Web serversC.Proceedings of the 3rd USENIX Windows NT Symposium,1999:155-164.2 Yang CS, Luo MY. Efficient support for content-based routing in web s
12、erver clustersC.Proceeding of the 2nd USENIX/IEEE Symposium on Internet Technologies and Systems,1999.3 徐忻,吴介一. Web服务结构模型的研究与实现J. 微计算机信息，2006，(5-3)：103-105。4 Cohen A, Rangarajan S, Slye H. On the performance of TCP splicing for URL-aware redirectionC. Proceedings of the 2nd USENIX Symposium on Inter
13、net Technologies and System, 1999:117-125.5 杜增凯.基于内容的分布式web服务器调度算法J.软件学报，2003，Vol.14,No.12.6 都志辉.高性能计算之并行编程技术MPI并行程序设计M.清华大学出版社 北京，2001第一版基金资助：国家重大基础研究规划项目(973)（编号G1999054000）；电子科技大学博士后基金作者：王燮（1970- ），男，汉族，讲师，研究方向为并行技术及应用。Email: 通讯地址：成都信息工程学院（航空港校区）软件工程系办公室 邮编610225王 鹏（1975- ），男，汉族，博士，电子科技大学博士后，2004
14、-2006年任中电集团某并行SAR地面数据站主持师，研究方向为并行技术及应用向 令（1983- ），男，土家族，工程师，研究方向为并行数值计算。黄松林（1986- ），男，汉族，工程师，研究方向为并行技术及应用。Hierarchical Web Server Cluster Based on MPIWANG Xie1 WANG Peng2, 1 XIANG Ling1 HUANG Song-lin1 (1.Parallel Computing Laboratory, Chengdu University of Information Technology, Sichuan, Chengdu,
15、610225; 2. University of Electronic Science and Technology of China, Sichuan, Chengdu, 610054)Abstract：A novel parallel web server cluster is proposed in our study. A hierarchical structure is adopted in this cluster. Each node of every layer is used to serve as scheduling server (except the last la
16、yer in this part) and data server. Data service is provided and judged by it according to pre-scheduled gauge. It can also diagnose the data service s work by the gauge and then transfer the overdone part to the node of next layer. A kind of monitor and control server is established based on ajax an
17、d vml technology to supervise every node timely. Key Words: Hierarchical Web Server Clusters；Measurement Threshold；Parallel File I/O；MPIEditors note: Judson Jones is a meteorologist, journalist and photographer. He has freelanced with CNN for four years, covering severe weather from tornadoes to typ
18、hoons. Follow him on Twitter: jnjonesjr (CNN) - I will always wonder what it was like to huddle around a shortwave radio and through the crackling static from space hear the faint beeps of the worlds first satellite - Sputnik. I also missed watching Neil Armstrong step foot on the moon and the first
19、 space shuttle take off for the stars. Those events were way before my time.As a kid, I was fascinated with what goes on in the sky, and when NASA pulled the plug on the shuttle program I was heartbroken. Yet the privatized space race has renewed my childhood dreams to reach for the stars.As a meteo
20、rologist, Ive still seen many important weather and space events, but right now, if you were sitting next to me, youd hear my foot tapping rapidly under my desk. Im anxious for the next one: a space capsule hanging from a crane in the New Mexico desert.Its like the set for a George Lucas movie float
21、ing to the edge of space.You and I will have the chance to watch a man take a leap into an unimaginable free fall from the edge of space - live.The (lack of) air up there Watch man jump from 96,000 feet Tuesday, I sat at work glued to the live stream of the Red Bull Stratos Mission. I watched the ba
22、lloons positioned at different altitudes in the sky to test the winds, knowing that if they would just line up in a vertical straight line we would be go for launch.I feel this mission was created for me because I am also a journalist and a photographer, but above all I live for taking a leap of fai
23、th - the feeling of pushing the envelope into uncharted territory.The guy who is going to do this, Felix Baumgartner, must have that same feeling, at a level I will never reach. However, it did not stop me from feeling his pain when a gust of swirling wind kicked up and twisted the partially filled
24、balloon that would take him to the upper end of our atmosphere. As soon as the 40-acre balloon, with skin no thicker than a dry cleaning bag, scraped the ground I knew it was over.How claustrophobia almost grounded supersonic skydiverWith each twist, you could see the wrinkles of disappointment on t
25、he face of the current record holder and capcom (capsule communications), Col. Joe Kittinger. He hung his head low in mission control as he told Baumgartner the disappointing news: Mission aborted.The supersonic descent could happen as early as Sunday.The weather plays an important role in this miss
26、ion. Starting at the ground, conditions have to be very calm - winds less than 2 mph, with no precipitation or humidity and limited cloud cover. The balloon, with capsule attached, will move through the lower level of the atmosphere (the troposphere) where our day-to-day weather lives. It will climb
27、 higher than the tip of Mount Everest (5.5 miles/8.85 kilometers), drifting even higher than the cruising altitude of commercial airliners (5.6 miles/9.17 kilometers) and into the stratosphere. As he crosses the boundary layer (called the tropopause), he can expect a lot of turbulence.The balloon wi
28、ll slowly drift to the edge of space at 120,000 feet (22.7 miles/36.53 kilometers). Here, Fearless Felix will unclip. He will roll back the door.Then, I would assume, he will slowly step out onto something resembling an Olympic diving platform.Below, the Earth becomes the concrete bottom of a swimmi
29、ng pool that he wants to land on, but not too hard. Still, hell be traveling fast, so despite the distance, it will not be like diving into the deep end of a pool. It will be like he is diving into the shallow end.Skydiver preps for the big jumpWhen he jumps, he is expected to reach the speed of sou
30、nd - 690 mph (1,110 kph) - in less than 40 seconds. Like hitting the top of the water, he will begin to slow as he approaches the more dense air closer to Earth. But this will not be enough to stop him completely.If he goes too fast or spins out of control, he has a stabilization parachute that can
31、be deployed to slow him down. His team hopes its not needed. Instead, he plans to deploy his 270-square-foot (25-square-meter) main chute at an altitude of around 5,000 feet (1,524 meters).In order to deploy this chute successfully, he will have to slow to 172 mph (277 kph). He will have a reserve p
32、arachute that will open automatically if he loses consciousness at mach speeds.Even if everything goes as planned, it wont. Baumgartner still will free fall at a speed that would cause you and me to pass out, and no parachute is guaranteed to work higher than 25,000 feet (7,620 meters).It might not
33、be the moon, but Kittinger free fell from 102,800 feet in 1960 - at the dawn of an infamous space race that captured the hearts of many. Baumgartner will attempt to break that record, a feat that boggles the mind. This is one of those monumental moments I will always remember, because there is no way Id miss this.