CN102104617A - Method for storing massive picture data by website operating system - Google Patents
Method for storing massive picture data by website operating system Download PDFInfo
- Publication number
- CN102104617A CN102104617A CN2010105720570A CN201010572057A CN102104617A CN 102104617 A CN102104617 A CN 102104617A CN 2010105720570 A CN2010105720570 A CN 2010105720570A CN 201010572057 A CN201010572057 A CN 201010572057A CN 102104617 A CN102104617 A CN 102104617A
- Authority
- CN
- China
- Prior art keywords
- picture
- server
- image data
- website
- servers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for storing massive picture data by a website operating system, which comprises the following steps of: separating picture servers from the website operating system, and performing centralized processing on user picture data storage operations by using the picture servers; and when a user logs in the system and starts storing pictures, finding a picture server cluster of the user according to an enterprise or user group of the login user, and according to the current loading conditions of each picture server of the picture server cluster, selecting the picture server with the lightest load to perform storage processing. The website operating system is separated from the picture serves, and massive pictures are distributed to each picture server by the distributed load balancing function of the picture servers, so the processing overpressure of a single picture server is avoided, the problem of storage bottleneck of a hard disk is solved and the website operating system can rapidly store the massive picture data.
Description
Technical field
The present invention relates to the method for a kind of website operation system storage mass picture data.
Technical background
Large-scale operation website, for example portal website faces a large number of users visit and high concurrent request every day, needs to handle the storage of mass picture data.For Website server, no matter be Apache, IIS or the server of other types, picture is consumption of natural resource.A large amount of concurrent storage demands of user's image data brings very big influence certainly will for the performance of system.Therefore, for the large-scale website operation system that the storage of mass users image data is arranged, be a long felt need for one and support the way of big data quantity picture storage to solve the systematic function bottleneck problem.
Summary of the invention
Main purpose of the present invention is to provide a kind of website operation system to handle the method for mass picture data, utilize distributed picture servers, realize that the picture processing separates with the page processing of website operation system, and, improve the ability of website operation system fast processing mass picture data by the picture servers load-balancing function.
The method of a kind of website operation system storage mass picture data, wherein,
At first, the website operation system is provided with several Website servers, and this Website server is connected with at least one picture servers group; This website operation system also connects one and stores the user grouping ID of user attaching or the database of enterprise's id information, picture servers essential information, picture servers group configuration information and picture-storage routing information;
User's Website login operation system uploading pictures is to Website server, arrive the user grouping ID or the ID of enterprise of the user attaching of this uploading pictures behind the described Website server reception image data according to database lookup, find corresponding picture servers group according to this user grouping ID or the ID of enterprise again, at last according to load-balancing algorithm, traversal is " the image data storage number of handling " and " the image data number of having handled " of all picture servers among this picture servers group relatively, picture servers by selection pressure minimum relatively is as the Target Photo server of accepting this picture-storage request, and this Website server submits to image data to finish picture-storage to the Target Photo server of correspondence;
Described Target Photo server receives image data and writes hard disk, after the picture-storage success, picture servers ID and picture-storage is returned to Website server and deposits database in the relative path and the picture file title thereof of destination server.
User's Website login operation system uploading pictures is to Website server, described Website server receives image data, whether judge the photo current size of data greater than the image data compression threshold,, then image data is compressed if greater than the image data compression threshold.
Load-balancing algorithm among the described picture servers group, concrete computational logic is as follows:
First picture servers among the picture servers group of step 1, taking-up user grouping ID or the ID of enterprise correspondence, if picture servers quantity is 0 to return failure, if picture servers quantity is 1, directly return first picture servers, if picture servers is greater than 1, interim preservation current server is " an interim selected server ", then goes to step 2;
Step 2, obtain next picture servers, if obtain failure, execution in step 3; If obtain success, then the picture servers with the front compares, if " the image data storage number of handling " of photo current server then continues execution in step 2 greater than " the image data storage number of handling " of " interim selected server "; If " the image data storage number of handling " of photo current server equals " the image data storage number of handling " of " interim selected server ", continue relatively " the image data number of having handled ", if " the image data storage number of having handled " of photo current server is less than " the image data storage number of having handled " of " interim selected server ", then with current picture servers as " interim selected server ", continue step 2; If " the image data storage number of having handled " of photo current server continues step 2 greater than " the image data storage number of having handled " of " interim selected server ";
Step 3, return " interim selected server " as Target Photo server of handling photo current storage request.
Because the present invention separates picture servers with the website operation system, focus on the storage operation of user's image data by picture servers.When logging in system by user begins picture, the website operation system is according to enterprise under the login user or user grouping, find out the picture servers group that this user should belong to, according to the loading condition of each picture servers in the photo current server zone, select the lightest picture servers of wherein load to carry out stores processor again.So, separate with picture servers by the website operation system, and the distributed load equalizing function of picture servers, mass picture is distributed to each picture servers, avoid separate unit picture servers processing pressure excessive and solved the problem of hard-disc storage bottleneck, so that this website operation system can quick storage mass picture data.
Description of drawings
Fig. 1 is picture servers logical architecture schematic diagram among the present invention;
Fig. 2 uploads the Stored Procedure schematic diagram for mass picture among the present invention.
The invention will be further described below in conjunction with the drawings and specific embodiments.
Embodiment
The method of a kind of website of the present invention operation system storage mass picture data specifically may further comprise the steps:
At first, as shown in Figure 1, the website operation system is provided with several Website servers, and this Website server is connected with at least one picture servers group; This website operation system also connects a database that stores relevant information, picture servers essential information, picture servers group configuration information and picture-storage routing informations such as the user grouping ID of user attaching or the ID of enterprise.
The webmaster reasonably disposes the quantity of picture servers according to the current site number of users, and depositing the picture servers essential information in database, described picture servers Basic Information Table mainly comprises: server ID, picture servers title, server address, operating state (0 read-write, 1 read-only 2 faults), founder, creation-time, reviser, modification time, remarks;
The online user's that the webmaster will have by user grouping or enterprise quantity disposes a plurality of picture servers, form the picture servers group, and deposit picture servers group configuration information in database, described picture servers group configuration information comprises: Record ID, user grouping or the ID of enterprise, picture servers ID, operating state (0 read-write, 1 read-only 2 faults), the founder, remarks, creation-time, the reviser, modification time.User grouping or enterprise can corresponding a plurality of picture servers, and a picture servers can corresponding a plurality of user grouping or enterprise.
Picture servers is according to the relative path of user grouping ID or ID of enterprise and picture-storage, calculate the absolute path of picture at the local hard drive of picture servers, and depositing the picture-storage routing information in database, described picture-storage routing information mainly comprises: picture relative path, picture name etc.
The website operation system reads various configuration informations and deposits local cache in from database, when user's Website login operation system and when proposing the picture-storage request, the website operation system reads the ID of enterprise or the user grouping ID of user's correspondence from local cache, obtain corresponding picture servers group according to the ID of this enterprise or user grouping ID again, calculate according to load-balancing algorithm at last and confirm that the Target Photo server is to finish picture-storage.
The website operation system can set in advance the image data compression threshold, the image data of accepting when system is greater than set threshold value, the website operating system is before sending image data to picture servers, can compress image data, with raising image data network transfer speeds, and then the picture disposal ability of raising system.
The flow process of user storage picture is as follows:
(1) user's Website login operation system, uploading pictures is to Website server, and described Website server receives image data, judges that whether the photo current size of data is greater than the image data compression threshold, if greater than the image data compression threshold, then image data is compressed;
(2) Website server finds the user grouping ID or the ID of enterprise of the user attaching of this uploading pictures according to database information, find corresponding picture servers group according to this user grouping ID or the ID of enterprise again, calculate the Target Photo server according to load-balancing algorithm at last, this Website server submits to image data to finish picture-storage to corresponding Target Photo server;
(3) described Target Photo server receives image data and writes hard disk, simultaneously, this picture servers return is replied (success, failure) and is given Website server, this Website server is given the user according to the operation acknowledge notifying operation result who returns, if store successfully, then picture servers ID and picture-storage are returned to Website server and deposit database at the relative path of Target Photo server, and return success operation acknowledge; If failure is then directly returned failure operation and is replied;
Because the webmaster disposes picture servers according to the user grouping or the enterprise of online user's ownership, so each online user has corresponding picture servers group, when an online user submits the picture request to, the website operation system will go out the user grouping ID or the ID of enterprise of this user attaching according to the information inquiry of database, and pairing picture servers group, relatively " the image data storage number of handling (WorkingImageCount) " and " the image data number of having handled (WorkedImageCount) " of all picture servers among this picture servers group of traversal then accepted this requested service device by the picture servers conduct of selection pressure minimum relatively.Concrete computational logic is as follows:
First picture servers among the picture servers group of step 1, taking-up user grouping ID or the ID of enterprise correspondence, if picture servers quantity is 0 to return failure, if picture servers quantity is 1, directly return first picture servers, if picture servers is greater than 1, interim preservation current server is " an interim selected server ", the operation below then continuing;
Step 2, obtain next picture servers, if obtain failure, execution in step 3; If obtain success, then compare with the front picture servers, if " the image data storage number of handling " of photo current server then continues execution in step 2 greater than " the image data storage number of handling " of " interim selected server "; If " the image data storage number of handling " of photo current server equals " the image data storage number of handling " of " interim selected server ", continue relatively " the image data number of having handled ", if " the image data storage number of having handled " of photo current server is less than " the image data storage number of having handled " of " interim selected server ", then with current picture servers as " interim selected server ", continue step 2; If " the image data storage number of having handled " of photo current server continues step 2 greater than " interim selected server " " the image data storage number of having handled ";
Step 3, return " interim selected server " as Target Photo server of handling photo current storage request.
The above, it only is preferred embodiment of the present invention, be not that technical scope of the present invention is imposed any restrictions, so every foundation technical spirit of the present invention all still belongs in the scope of technical solution of the present invention any trickle modification, equivalent variations and modification that above embodiment did.
Claims (3)
1. the method for website operation system storage mass picture data is characterized in that:
At first, the website operation system is provided with several Website servers, and this Website server is connected with at least one picture servers group; This website operation system also connects one and stores the user grouping ID of user attaching or the database of enterprise's id information, picture servers essential information, picture servers group configuration information and picture-storage routing information;
When user's Website login operation system uploading pictures arrives Website server, arrive the user grouping ID or the ID of enterprise of the user attaching of this uploading pictures behind the described Website server reception image data according to database lookup, find corresponding picture servers group according to this user grouping ID or the ID of enterprise again, at last according to load-balancing algorithm, traversal is " the image data storage number of handling " and " the image data number of having handled " of all picture servers among this picture servers group relatively, picture servers by selection pressure minimum relatively is as the Target Photo server of accepting this picture-storage request, and this Website server submits to image data to finish picture-storage to the Target Photo server of correspondence;
Described Target Photo server receives image data and writes hard disk, after the picture-storage success, picture servers ID and picture-storage is returned to Website server and deposits database in the relative path and the picture file title thereof of destination server.
2. the method for a kind of website according to claim 1 operation system storage mass picture data, it is characterized in that: when user's Website login operation system uploading pictures arrives Website server, described Website server receives image data, judge that whether the photo current size of data is greater than the image data compression threshold, if greater than the image data compression threshold, then image data is compressed.
3. the method for a kind of website according to claim 1 operation system storage mass picture data is characterized in that load-balancing algorithm among the described picture servers group, specifically comprises the steps:
First picture servers among the picture servers group of step 1, taking-up user grouping ID or the ID of enterprise correspondence, if picture servers quantity is 0 to return failure, if picture servers quantity is 1, directly return first picture servers, if picture servers is greater than 1, interim preservation current server is " an interim selected server ", then goes to step 2;
Step 2, obtain next picture servers, if obtain failure, execution in step 3; If obtain success, then the picture servers with the front compares, if " the image data storage number of handling " of photo current server then continues execution in step 2 greater than " the image data storage number of handling " of " interim selected server "; If " the image data storage number of handling " of photo current server equals " the image data storage number of handling " of " interim selected server ", continue relatively " the image data number of having handled ", if " the image data storage number of having handled " of photo current server is less than " the image data storage number of having handled " of " interim selected server ", then with current picture servers as " interim selected server ", continue step 2; If " the image data storage number of having handled " of photo current server continues step 2 greater than " the image data storage number of having handled " of " interim selected server ";
Step 3, return " interim selected server " as Target Photo server of handling photo current storage request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105720570A CN102104617A (en) | 2010-11-30 | 2010-11-30 | Method for storing massive picture data by website operating system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105720570A CN102104617A (en) | 2010-11-30 | 2010-11-30 | Method for storing massive picture data by website operating system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102104617A true CN102104617A (en) | 2011-06-22 |
Family
ID=44157143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010105720570A Pending CN102104617A (en) | 2010-11-30 | 2010-11-30 | Method for storing massive picture data by website operating system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102104617A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103347062A (en) * | 2013-06-24 | 2013-10-09 | 北京汉柏科技有限公司 | Virtual computing resource distribution method |
| CN103488644A (en) * | 2012-06-12 | 2014-01-01 | 联想(北京)有限公司 | Method for carrying out data storage and database system |
| CN103731448A (en) * | 2012-10-11 | 2014-04-16 | 腾讯科技（深圳）有限公司 | Method and system for compressing picture |
| CN105187491A (en) * | 2015-08-06 | 2015-12-23 | 上海斐讯数据通信技术有限公司 | Method, system and server thereof for uploading pictures |
| CN105847415A (en) * | 2016-05-12 | 2016-08-10 | 广西尊达电子商务有限公司 | Distributed server system |
| CN106844020A (en) * | 2016-12-01 | 2017-06-13 | 明博教育科技股份有限公司 | Different task travels through the high-efficient treatment method and system of a large amount of shared datas |
| CN106982247A (en) * | 2017-02-23 | 2017-07-25 | 成都谷辘信息技术有限公司 | A kind of picture distributed memory system based on WEB |
| CN106998487A (en) * | 2017-04-27 | 2017-08-01 | 努比亚技术有限公司 | image processing method, device and mobile terminal |
| CN108347465A (en) * | 2017-01-23 | 2018-07-31 | 阿里巴巴集团控股有限公司 | A kind of method and device of selection network data center |
| CN110351312A (en) * | 2018-04-02 | 2019-10-18 | 腾讯科技（深圳）有限公司 | Picture sharing method, device, computer equipment and storage medium |
| CN111159441A (en) * | 2019-12-25 | 2020-05-15 | 北大方正集团有限公司 | Picture editing method, picture editing apparatus, and computer-readable storage medium |
| CN112003956A (en) * | 2020-10-27 | 2020-11-27 | 武汉中科通达高新技术股份有限公司 | Traffic management system |
| CN113453025A (en) * | 2020-03-26 | 2021-09-28 | 杭州海康威视系统技术有限公司 | Data acquisition method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1794239A (en) * | 2005-12-30 | 2006-06-28 | 张天山 | Automatic generating system of template network station possessing searching function and its method |
| CN1972311A (en) * | 2006-12-08 | 2007-05-30 | 华中科技大学 | A stream media server system based on cluster balanced load |
| CN101510218A (en) * | 2009-03-26 | 2009-08-19 | 阿里巴巴集团控股有限公司 | Method for implementing picture search and website server |
| US20100115070A1 (en) * | 2008-05-27 | 2010-05-06 | Thierry Missimilly | Method for generating manipulation requests of an initialization and administration database of server cluster, data medium and corresponding a server cluster, data medium and corresponding service cluster |
| CN101854388A (en) * | 2010-05-17 | 2010-10-06 | 浪潮(北京)电子信息产业有限公司 | Method and system for parallel access to a large number of small files in cluster storage |
| CN101866359A (en) * | 2010-06-24 | 2010-10-20 | 北京航空航天大学 | A small file storage and access method in a cluster file system |
-
2010
- 2010-11-30 CN CN2010105720570A patent/CN102104617A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1794239A (en) * | 2005-12-30 | 2006-06-28 | 张天山 | Automatic generating system of template network station possessing searching function and its method |
| CN1972311A (en) * | 2006-12-08 | 2007-05-30 | 华中科技大学 | A stream media server system based on cluster balanced load |
| US20100115070A1 (en) * | 2008-05-27 | 2010-05-06 | Thierry Missimilly | Method for generating manipulation requests of an initialization and administration database of server cluster, data medium and corresponding a server cluster, data medium and corresponding service cluster |
| CN101510218A (en) * | 2009-03-26 | 2009-08-19 | 阿里巴巴集团控股有限公司 | Method for implementing picture search and website server |
| CN101854388A (en) * | 2010-05-17 | 2010-10-06 | 浪潮(北京)电子信息产业有限公司 | Method and system for parallel access to a large number of small files in cluster storage |
| CN101866359A (en) * | 2010-06-24 | 2010-10-20 | 北京航空航天大学 | A small file storage and access method in a cluster file system |
Non-Patent Citations (1)
| Title |
|---|
| 李文中 等: "服务组合中一种自适应的负载均衡算法", 《软件学报》, no. 5, 31 May 2006 (2006-05-31) * |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103488644A (en) * | 2012-06-12 | 2014-01-01 | 联想(北京)有限公司 | Method for carrying out data storage and database system |
| CN103488644B (en) * | 2012-06-12 | 2017-12-15 | 联想(北京)有限公司 | Carry out the method and Database Systems of data storage |
| CN103731448B (en) * | 2012-10-11 | 2017-09-26 | 腾讯科技（深圳）有限公司 | A kind of method and system of compressed picture |
| CN103731448A (en) * | 2012-10-11 | 2014-04-16 | 腾讯科技（深圳）有限公司 | Method and system for compressing picture |
| WO2014056292A1 (en) * | 2012-10-11 | 2014-04-17 | 腾讯科技（深圳）有限公司 | Image compression method and system |
| US9432672B2 (en) * | 2012-10-11 | 2016-08-30 | Tencent Technology (Shenzhen) Company Limited | Image compression method and system with image compression time information |
| CN103347062A (en) * | 2013-06-24 | 2013-10-09 | 北京汉柏科技有限公司 | Virtual computing resource distribution method |
| CN105187491A (en) * | 2015-08-06 | 2015-12-23 | 上海斐讯数据通信技术有限公司 | Method, system and server thereof for uploading pictures |
| CN105847415A (en) * | 2016-05-12 | 2016-08-10 | 广西尊达电子商务有限公司 | Distributed server system |
| CN106844020A (en) * | 2016-12-01 | 2017-06-13 | 明博教育科技股份有限公司 | Different task travels through the high-efficient treatment method and system of a large amount of shared datas |
| CN108347465A (en) * | 2017-01-23 | 2018-07-31 | 阿里巴巴集团控股有限公司 | A kind of method and device of selection network data center |
| CN108347465B (en) * | 2017-01-23 | 2021-02-02 | 阿里巴巴集团控股有限公司 | Method and device for selecting network data center |
| CN106982247A (en) * | 2017-02-23 | 2017-07-25 | 成都谷辘信息技术有限公司 | A kind of picture distributed memory system based on WEB |
| CN106998487A (en) * | 2017-04-27 | 2017-08-01 | 努比亚技术有限公司 | image processing method, device and mobile terminal |
| CN110351312A (en) * | 2018-04-02 | 2019-10-18 | 腾讯科技（深圳）有限公司 | Picture sharing method, device, computer equipment and storage medium |
| US11483414B2 (en) | 2018-04-02 | 2022-10-25 | Tencent Technology (Shenzhen) Company Limited | Picture sharing method, computer device, and storage medium |
| CN110351312B (en) * | 2018-04-02 | 2021-10-26 | 腾讯科技（深圳）有限公司 | Picture sharing method and device, computer equipment and storage medium |
| CN111159441A (en) * | 2019-12-25 | 2020-05-15 | 北大方正集团有限公司 | Picture editing method, picture editing apparatus, and computer-readable storage medium |
| CN113453025A (en) * | 2020-03-26 | 2021-09-28 | 杭州海康威视系统技术有限公司 | Data acquisition method and device |
| CN113453025B (en) * | 2020-03-26 | 2023-02-28 | 杭州海康威视系统技术有限公司 | Data acquisition method and device |
| CN112003956B (en) * | 2020-10-27 | 2021-01-15 | 武汉中科通达高新技术股份有限公司 | Traffic management system |
| CN112003956A (en) * | 2020-10-27 | 2020-11-27 | 武汉中科通达高新技术股份有限公司 | Traffic management system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102104617A (en) | Method for storing massive picture data by website operating system | |
| US10776396B2 (en) | Computer implemented method for dynamic sharding | |
| US20200278985A1 (en) | Content delivery network analytics management via edge stage collectors | |
| US10223431B2 (en) | Data stream splitting for low-latency data access | |
| US9609050B2 (en) | Multi-level data staging for low latency data access | |
| CN105025053A (en) | Distributed file upload method based on cloud storage technology and system | |
| CN111885098B (en) | Proxy access method, system and computer equipment for object storage cluster | |
| CN103338252B (en) | Realizing method of distributed database concurrence storage virtual request mechanism | |
| CN102460398A (en) | Source classification for performing deduplication in a backup operation | |
| CN101753478A (en) | Distributed E-mail system and service method thereof | |
| CN105159845A (en) | Memory reading method | |
| CN106202416A (en) | Table data write method and device, table data read method and device | |
| US20230109530A1 (en) | Synchronous object placement for information lifecycle management | |
| US20240372932A1 (en) | Request processing in a content delivery framework | |
| CN107612950A (en) | A kind of method, apparatus, system, electronic equipment that service is provided | |
| CN112559459B (en) | Cloud computing-based self-adaptive storage layering system and method | |
| JP5371656B2 (en) | File search system | |
| CN106506647A (en) | A kind of client has the intelligence community cloud storage system of data backup device | |
| CN103106242A (en) | Phone bill query method and phone bill query system | |
| CN102024037A (en) | Retrieval method of massive image data of website operation system | |
| CN113688115A (en) | File big data distributed storage system based on Hadoop | |
| CN103369368A (en) | Video cloud on-demand cache scheduling method supporting multi-code-rate version | |
| KR20160145250A (en) | Shuffle Embedded Distributed Storage System Supporting Virtual Merge and Method Thereof | |
| JP6110354B2 (en) | Heterogeneous storage server and file storage method thereof | |
| CN103368854B (en) | The Notification Method of a kind of RSCN message and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110622 |