SDSCDatacenterReferenceGuide

[Link]: Thepurposeofthisdocumentistoserveasreferencematerialforcampuspersonnelandcustomers [Link] madeasnecessarytomaintainitsaccuracy. [Link]: SDSCDataCenterAttributes: 19,[Link] 4Megawattsofcurrentdatacenterpower,withonsitecapacityof13Megawatts o 208Vdistributioncurrentlyavailable o 480Vavailable

EquipmentRacks: TherackstobeusedandprovidedintheSDSCdatacentersareAPCAR3100sintheWestDatacenterand Liebert/KnurrMiracel600mmwidth,1200mmdepth,[Link] [Link] troughsandthermalcontainmentsystems. [Link] [Link] Networking: ThedesignoftheSDSCcolocationnetworkfabricprovidesastable,flexibleinfrastructurethathashigh performanceandreliability.Allbackbonetrunkscurrentlyutilize10glinks,insomecasesusingmore thanonebondedtogetherforincreasedthroughput.TheLayer2/3ColofabricwillincludeJunipercore andAristaaggregationswitches.Thedesignwillsupportthousandsof1Gportsandhundredsof10g [Link] [Link] [Link] dependinghowacustomerwantstousetheirrackspace. BelowistheagreeduponSDSCnetworkinfrastructuretosupportscalablecolocation:

RevisedAugust2009

 Ataracklevel,therearethreestandardconnectivityoptions: Inrack48or24portJuniperEX4200switches: o Theswitchisrechargedtothecampusupfrontwithanannualmaintenancecost chargedformaintenanceandthedepreciationoftheupstreamfabric. o Hostsareconnectedtooneortwo1Gcopperportsortoa10Gbportonthe4200 switch. o UptotenEX4200switchessharetrunksbackintothefabricusingaredundant128Gb ringarchitecturethatallowalltenswitchestoactasoneswitchfromamanagement andperformanceperspective. 10Gblinksdirectlyintotheaggregationswitchfabric: o Forhigherbandwidthneedsorotherspecialcircumstances,10Gblinkscanbeprovided intotheAristaaggregationswitchlayerofthenetworkfabric. o These10Gbportsareleasedonanannualbasis,recoupingthedepreciationcostsof theirshareoftheupstreamfabric. 10Gblinksdirectlyintothecoreswitchfabric: o Forcertainspecialcircumstances,10GblinkscanbeprovideddirectlyintotheJuniper coreswitchlayerofthenetworkfabric. o These10Gbportsareleasedonanannualbasis,recoupingthedepreciationcostsof theirshareoftheupstreamfabric.

RevisedAugust2009

Additionalspecialconfigurationsareavailableonacasebycasebasisworkingwiththenetworking [Link] peerpoint,dedicated10GL2/L3path(s)toCENIC,orotherWANnetworkingneeds. IfusingSDSCaddressspace,[Link] providedsubnetsandVLANsasneededtosupporttheirnetworkingneeds,withtheallocatedamountof [Link] [Link](butnotall)cases,DNSservicesmaybeabletobedelegatedtotheparticipatingcampus. Alternately,campusdelegatedaddressspacewouldbecompletelymanagedbytheparticipating campusandnotincuranyperIPcosts. Additionalservicesinclude: TimesuppliedbySDSCservers. SNMPstatsonacolocationwebserverofVLANusageorportusageattheheadswitchinarack. PerformancetestingNDTserversonlineforlocalorremotetestingofbandwidth. Troubleshooting/configurationsassistanceatcurrenthourlyrates. Prohibitedactivitiesinclude: NATIPaddressingfordataaccess(VPNaccessispermitted). SpanormirrorportsinthefabricswitcheswithoutcoordinationwithSDSCnetworking. Excessiveperformancetestingthroughfabric. RackPower: SDSCwillprovidetwoNEMAL630,30Amp,[Link] powerisrequired,alternatearrangementsshouldbemadewithSDSC. Conditionedpowercouldbeavailablebutisnotguaranteed.

SeismicProtection: Allequipment([Link],mainframes,clusters,etc.)placedintheSDSCDataCenterwillneedto [Link] eventofanearthquakeorothermajorcatastrophe. o [Link]

AisleContainment: SDSCutilizesaislecontainmenttomaximizelifeandefficiencyofDatacenterequipment. EquipmenthostedattheSDSCDatacentermusthaveanairfloworientationfromfronttoback.

[Link]: HVAC:

RevisedAugust2009

WestDatacenter: o TheSanDiegoSupercomputerCenterutilizesLiebertairhandlingunitsforDatacenter cooling.IntheWestDatacentertherearemultiple30and40toncoolingunitsplaced [Link] [Link],sending theexhaustedhotairbacktotheCRAC(computerroomairconditioning)[Link] [Link] thissupplyisinterrupted,SDSChasalocalchilledwaterloopthatcansupply supplementalcoolingduringtheoutage. EastDatacenter: o [Link] [Link] [Link] systemisperformedinsidetheSDSCNOC.

FireSuppression: Detection: o Thedetectionsystemisacombinationofinroomandunderfloorparticledetection devices. WaterSuppression: o LikemostbuildingfacilitiestheSDSCDatacenterisequippedwithawetfireextinction [Link],meaningthepipesarenotfilledwithwateruntilinitial [Link] [Link] criticaltemperature. o Afalsepositivecanoccurifoneofthenozzlesrecessedintheceilingisdamagedor [Link] guidelinesshouldbefollowedalltimesintheSDSCDatacenter. WestDatacenterCleanAgent(Halon): o Thedetectorsaresimilartotheonesusedinthewatersystemhowevertheyare [Link] [Link] arerequiredtodischargethesystem. o ThereisanabortbuttonphysicallylocatedintheDatacenterthatcanpreventdischarge ifpressedbeforetheseconddetection. EastDatacenterHFC125(FM200CleanAgenttype): o [Link] [Link] [Link], thereisanabortbuttonlocatednearthemainentrancetotheDatacenternexttothe EPObutton.

RevisedAugust2009

Oncethefiresuppressionsystemenergizes(eitherwaterorHalon),theroomwillEPO (emergencypoweroff)allequipmentexceptforthelights(forsafetypurposes).Thereare6 EPObuttons(4intheexistingDatacenterand2moreintheexpansion)throughoutthe Datacenterthatcanalsobemanuallypushedintheeventofanemergency.

Power&Electricity: UtilitySupply: o SDSCcurrentlyhastwoutilityprovidersthatsupplyelectricitytothebuildingand [Link],thesesystemstotal ~12000kVa. Distribution: o SDSChasmultipletransformersandpowerdistributionunits(PDU)thatwillsupply~10 megawattsofpowertotheDatacenter.Distributionvoltagesrangefrom110to480. UninterruptablePowerSupply(UPS): o [Link] UPSprimarilykeepcoreinfrastructureandcriticalsystemsrunningintheeventofa [Link] SDSCNOC,andtheSanDiegoNetworkAccessPoint(SDNAP) Generator&CatastropheSupport: o TherearetwogeneratorsthatsupplytheDatacenterwithsupplementalpowerinthe [Link] withanearbyfacilitybutdoessupplyasubstantialamountofenergytotheDatacenter [Link] FacilitiesManagementregularlyforproperoperationandcondition. o [Link] thattheutilitysupplyislost,thenaturalgasplantcansupplytherequiredenergyfor nearregularoperation.

Security: PhysicalAccess: o CCTVSecurityCameraSystem: SDSCcurrentlyhassecuritycamerasfocusedonallentryandexitpointsofthe [Link] tohallways,publicspaces,andthephysicalexteriorofthehostbuildingtothe Datacenter. Therecordedfootageisstoreddigitallyandsecurelyandisretainedfora [Link],itisreadily availableintheSDSCNetworkOperationsCenter(NOC). o PhysicalAccesstothebuildingandtheDatacentercanbesegregatedthroughtheuseof [Link] thevascularpatternofyourhandforidentityverification.

RevisedAugust2009

o o

Duringnormalbusinesshoursof8amto5pmPDT,SDSCslobbydoorsareunlockedand [Link],thedoorsarelockedandrequirebiometricaccessoran [Link] 24x7x365. IntheeventthatapersonattemptingtoenterSDSCisnotenrolledintheIdentiscan system,[Link] phonescalldirectlytotheSDSCNOC,whereanondutyoperatorisresponsiblefor verifyingandauthorizingaccess. EnrollmentforSDSC/DatacenterAccess: Bydefault,alltechnicalpointsofcontact(TPOC)specifiedoneachSLAwillbe enrolledintheIdentiscansystemandconsideredbySDSCauthorized representativesofthecustomer/campus. Theenrollmentprocessrequirestheindividualtobephysicallypresentatthe [Link]. Authorizationbythecustomer: Priortotheindividual(s)arrivingattheSDSCDataCenter,thecustomer shouldauthorizeaccessthroughoneofthesechannels: o Anemailsenttooperator@[Link], contactinformation,thenameoftheindividual(s)toallow entry,andanyspecialinstructionstoSDSCNOCstaff(e.g. certainhoursofaccess,etc.). o CalltheSDSCNOCat858.534.5090toverballyauthorizethe individual(s).ThepasscodechosenduringtheSLAwillneedto [Link] SDSCNOCindefinitelyforverificationpurposes. Duringtheenrollmentprocessthefollowinginformationiscollected:fullname, emailaddress,aphotographtobestoredwiththeenrollmentrecord,acurrent phonenumber,thecustomerbeingrepresented([Link],UCLA,etc.),andany specifichourstopermitaccessifrequestedbythecustomer. Tocompletetheregistrationprocess,theenrolleeuseseitherhandforvascular recognitionandcreatesauniquePIN. ***AvalidformofIDmustbepresentedattimeofenrollmentfor [Link] PII. IntheeventtheIdentiscanaccesssystemisofflineuseofacallboxisrequired. AccesstotheSDSCfacilitywithoutIdentiscanenrollment: AnyonerequestingaccesstotheSDSCDatacenterwhoisnotenrolledinthe [Link] contractorsorspecialstaffneededintheeventofanoutage. [Link](s) requestingaccesswillberequiredtoshowphotoidentificationandto

RevisedAugust2009

signinontheSDSCDatacenterAccessLog,locatedjustoutsidethe SDSCNOC. EachtimeapersonwishestoentertheDatacenter,anotherentryon theSDSCAccessLogisrequired.

SystemandNetwork: o SDSCsnetworkandsecuritypracticesaremanagedbySDSCstaffunderadifferent [Link],fordetectingsystem vulnerabilities,networkmonitoringandintrusiondetection/preventionisutilized. GeneralpracticesdospanacrossbothSDSCandUCSDbutarenotidentical. o Systemsecurityistheresponsibilityofthecustomerunlessotherwisespecifiedinthe appropriateSLA.(E.g.:ThesystemismanagedbySDSCforanadditionalfee) o Toensuretheavailability,securityandintegrityoftheSDSCDatacenter,systemslocated intheDatacenteraresubjecttonetworksecuritymonitoringbytheSDSCsecurityteam. o SDSCwillscanthenetworkforvulnerablesystemsandcompromisedsystemsona [Link] [Link] schedulinganddepthofthescan. o Therearetwoclearsecurityincidentscenarios: SuspectedVulnerability:IftheresultsfromtheSDSCscanshowvulnerabilities existonahost,thecampustechnicalpointofcontactwillbeemailedwith detailedinformationincluding: Detectiondate Vulnerabilitytype Reportinformationgeneratedbythescan. Ifremediationofavulnerabilitythatcouldleadtoacompromiseisnot detectedbythenextscheduledscan,thehost(s)maybedisconnected fromthenetwork. SuspectedCompromise:Ifasystemissuspectedtobecompromised,SDSC reservestherighttodisconnectthesystemfromthenetworktoprevent [Link],anemailnotificationwillbesenttothe contactsspecifiedintheSLAwiththefollowinginformation: Timeofdisconnect Areportofaffectedsystem(s)includingwhichvulnerabilitieswere detected. TheFootprintsticketnumberassociatedwiththeincident. Toprotectothersystemsandthenetwork,SDSCmayrequestfromthe technicalPOCthataforensicimageofthesystembecreatedforfurther [Link]. o Itisthecampustechnicalpointofcontactsresponsibilitytocoordinateallactivities [Link] [Link]

RevisedAugust2009

technicalpointofcontactwillberesponsibleforensuringissuesareremediatedandany ticketsarecommunicatedasclosedtoSDSC. o SDSCcolocationminimumsecuritystandardspertainingtosensitive(PII/PHI/HIPAA) data: ****TheseminimumsecuritystandardsareSDSCrecommendedanddonot [Link] regardtobothphysicalsecurityandtechnicalsecurityistheresponsibilityofthe customerandtheirprogramauthority. Differentadministrativeentities(departments,groups,etc.)willnotoccupythe samephysicalracksunlessalladministrativeentitiesagreetothespacesharing [Link] physicalaccesstothemachinesbyotherentitiesintheracksanddesigneesof thoseentities(vendors,administrators,etc.).Thisagreementmustbeclearly statedontheServiceAgreementsofallentitiesintherackbeforethe installationofanyequipment. Ifmorethanonegroup/departmentwantstoshareoccupancy,theyare responsibleforensuringthatshared(andtherebyonlypartiallycontrolled) accessisacceptablebytheprojectsponsor...[Link],CMS,NSF,etc. Systemscontainingsensitiveinformationmustresideinracksthathavehandle [Link]$300costtoprocureandinstallthe locksontherack(APCAR8132A),andthisadditionalcostwillberechargedto theuseroftherack(notincludedinthebasecoloservice). Systemscontainingconfidentialinformationmustensurethatthedatais encryptedbeforetransitto/fromthesystem. Formoreinformationregardingspecificpolicies,pleasecontactSDSCsChief SecurityOfficer.

[Link]&EQUIPMENTMANAGEMENT: Shipping&Receiving: SDSCsloadingdockcanaccommodateallvehiclesincludingastandardcommercialtractor [Link] notificationandassistancepurposes. [Link] orrequiredtoperformthisservice,timeandmaterialswillbepaidforbythecustomer. Incoming:

RevisedAugust2009

WhenacustomershipsequipmenttoSDSC,itistheirresponsibilitytoarrangeshipment withthecarrier. o Oncetrackinginformationhasbeenreceived,itshouldbeemailedtoSDSC (operator@[Link])withthefollowinginformation: Customername Campusororganizationname Nameofcarrier Numberofpackages Trackingnumbersforallcontainers o Uponreceiptofthedelivery,SDSCwillinventoryandassesstheconditionofthe [Link],SDSCwilldecline thedelivery. o Iftheshipmentiscompleteandtheconditionverified,SDSCwillacceptthecontainers [Link]:one [Link] thesespacesisenvironmentallycontrolled. o SDSCwillnotbeheldliableforanydamagedormissingitemsduringshipping. Outgoing: o WhenequipmentisscheduledtoleaveSDSC,itistheresponsibilityofthecustomerto [Link] equipmentforshipmentadheringtoSDSCDatacenterpoliciesandsafetyrules. o Oncethepickupisscheduled,thecustomerwillemailSDSC(operator@[Link])the followinginformation: Customername Campusororganizationname Nameofcarrier Numberofpackages Trackingnumbersforallcontainers o AftertheshipmentleavesSDSC,aconfirmationemailwillbesenttothecustomerand TPOCspecifiedonthecorrespondingSLA. o

CommissioningEquipment: InaccordancewiththecurrentSDSCDatacenterpolicy,packingmaterialsarenotpermitted [Link] [Link] disposalofallpackingmaterials. PriortotheinstallationofequipmentintheSDSCDatacenter,thefollowinginformationwill needtobesenttoSDSC(operator@[Link]): o Date(s)ofinstallation o EquipmentlistifdifferentfromtheoneprovidedwiththeSLA. o Anyadditionalnetworkingneeds.

RevisedAugust2009

o Anyadditionalmaterialsneededfortheinstallation([Link],etc.). o AnyworkspecificationstobecontractedtheSDSCtocompletetheinstallation. EquipmentbeingcommissionedintheSDSCDatacenterwillbeenteredintotheinventory [Link] [Link] informationwillbestoredintheSDSCinventorysystem. Isobaseseismicplatformswillbeorderedpriortoinstallation.

DecommissioningEquipment: Priortothedecommissioningofequipment,thecustomermustcontactSDSC (operator@[Link])withthefollowinginformation: o Date(s)ofplanneddecommissioning(SDSCwillneedtobenotifiedofanychange) o Equipmentbeingremoved(byeitherrackunit,racklocation,orsystemname) o Name(s)ofauthorizedpersonnelperformingtheworkonbehalfofthecustomer WhentheequipmentisremovedfromtheDatacenteritsstatusintheSDSCinventorysystem [Link] equipmentatthetimeofinstallation. AllSDSCprovidedcablingshouldbereturnedtotheSDSCNOCatthistime. AllpackingofequipmentmusttakeplaceoutsideoftheSDSCDatacenterinaccordancewiththe [Link],ladders, carts,andhandtoolstoproperlydismountordisconnectallequipmentfromtheDatacenter. Packingofequipmentcantakeplaceineitherofthetwopreviouslyspecifiedstoragelocations.

ColocationRequestProcess: NewcolocationrequestsarereceivedbythecurrentSDSCDatacenterManagerasatechnical [Link] [Link],theticketisreassigned [Link] foranycontractedtechnicalassistance(ifany).Theactualprocessforhandlinganew colocationrequestattheSDSCDatacenter:

RevisedAugust2009

 Cabling: [Link]. Allcableswillbelabeledwiththefollowinginformation: o Networkcable:SwitchName/Switchportnumber/Hostname(e.g.itn77swGi0/2 labyrinth) [Link] willberesponsibleforanycostinreplacinglostordamagedcables. SDSCNOCstaffwillinspectcustomercablingpracticesatthetimeofequipmentinstallation. [Link] meetSDSCstandards,[Link] thecustomerorSDSCNOCstaffforanadditionalfee.

RevisedAugust2009

[Link] eventofnoncompliance,thetechnicalpointofcontactwillbecontactedviaemailandanSDSC Footprintsticketcreateduntilremediationiscomplete.

EquipmentMountinginRacks: [Link] maintenancepurposes.

OSInstallation&Maintenance: SDSCdoesnotincludeoperatingsysteminstallationandmaintenanceinthebasicservice. AdditionalOSmaintenanceservicesareavailable(refertotheSLAforadditionalservice specifications).

[Link],MONITORING,&SUPPORT Hours&ContactInformation: SDSCOperations:24x7,365daysayear o Contact:operations@sdsc.eduorcall858.534.5090 SDSCBusinesshours: o MondayFriday:8am5pmPDT. o SaturdaySunday:closed. DepartmentContact: MattCampbell DataCenterServicesManager mattc@[Link] ph:858.361.8343

Support: SDSCNOCstaffwillonlyprovideremotesupportspecifiedintheSLA(ifapplicable). o Intheeventremotehandshasbeenauthorized,anyperson(s)requestingSDSCNOC [Link] codeisnotcorrect,therequestwillbedenied. o SDSCNOCstaffwillcreateaticketintheSDSCFootprintsticketingsystemtorecordand [Link](s)willreceiveanemailuponthecreationof theticketandshouldusethisastheprimarymeansofcommunicationfortherequest. o AllFootprintsticketswillbekeptforauditingandreviewpurposes. o IfnoSDSCNOCremotehandsservicehasbeenestablished,allrequestswillbedenied.

Notifications&Outages:

RevisedAugust2009

Plannedmaintenance(performedbySDSC):2weekaverageannouncementpriortoworkbeing performed. o Allattemptsmadetoperformoutsideofnormalbusinesshours(seeabove). Unplannedoutageoremergency:AlleffortswillbemadebySDSCanditspartnerstoreturn [Link] tocustomersasappropriate. Allnoticesaresenttoopsnotice@[Link] o Subscriptiontothismaillistismandatoryforalltechnicalpointsofcontact(TPOC) [Link] todatebynotifyingSDSCOperationsofanypersonnelchanges.

Monitoring: TheSDSCNetworkOperationsCenter(NOC)hasanumberofrealtimemonitors: o Network:TheNOCutilizesnetworkmonitoringtoolsmanagedbySDSCsEnterprise [Link] connections,butthemainlinkstotheUCSDcampusandmainuplinkstoCENIC,ESNet, etc. o SystemsandNetworkSecurity:TheSDSCSecurityteamutilizessecuritymonitoringtools toprotectagainst,[Link] toolsalsopermittheSecurityStafftonotifyacampusstechnicalpointofcontactinthe eventasystemispotentiallycompromised. o BuildingSecurity:TheCCTVsecuritycamerasystemsaremonitored24x7x365inthe NOC. o Facilities:Thereareautomatedbuildingmonitorsforthehostbuildingaswellasthe DataCentertoalertNOCpersonnelandUCSDFacilitiesManagementifanyofthe [Link] dailytomaintainthefacilitiesintegrity. o UPS:[Link] [Link] thattimereplacementpartsorserviceisscheduled. o Temperature:[Link] thresholdtemperatureisexceeded,NOCstaffarenotified.

ServiceDefinitions: o SystemAdministration:SDSCSystemAdministrationisavailableonanhourlybasisand [Link] maintenanceisperformedonceamonthataspecifictimeunlessacriticalvulnerability [Link]. SecurityServices:TheSDSCSecurityteamisavailableonanhourlybasistosupportall areasofsystemsecurity,includingOShardening,IntrusionDetectionSystem,firewall

RevisedAugust2009

o o

securityconfiguration,antivirus,auditing,intrusionprotection,forensics,and penetrationtestingandsystemlockdown. BackupServices:SDSChasanumberofbackupoptionsinplacetosuitdifferentlevels [Link],Linux,[Link] [Link] configurablebasedoncampusrequirements. Outofbandremoteaccess:Anyequipmentforoutofbandmanagementisprovided byeachparticipatingcampus. AssistedInstallationandRemoteHands:SDSCOperationsstaffassistanceisavailable. Staffareexperiencedininstallingsystemandnetworkingequipment,includingrack mountingandcablingandoperatingsysteminstallation.

[Link]&CONDUCT: RefertotheServiceAgreementforcurrentSDSCDatacenterconductpolicies.

[Link]: BillingCycle: SDSCsinvoicesaresentmonthly.

SampleInvoice: [Link] itemized.

RevisedAugust2009