Hewlett Packard Enterprise Gen10 Servers With Amd Epyc 7002 And 7003-series Processors Instructions

Hewlett Packard Enterprise Gen10 Servers With Amd Epyc 7002 And 7003-series Processors Instructions

Hewlett LOGOSERVER MEMORY POPULATION RULES
FOR HPE PROLIANT GEN10 AND GEN10
PLUS SERVERS WITH AMD EPYC 7002
AND 7003-SERIES PROCESSORS
Technical white paper

Check if the document is available in the language of your choice.Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors

INTRODUCTION

This paper provides an overview of HPE DDR4 Smart Memory and its use in HPE ProLiant servers using the AMD EPYC™ 7002 and 7003 processor family. HPE  ProLiant servers with AMD EPYC 7002 and 7003 introduce HPE DDR4-2933 and HPE DDR4-3200 memory, which has faster data rates, lower latencies, and greater power  efficiency than the memory used in previous generations of HPE ProLiant servers.
HPE Smart Memory also provides superior performance over third-party memory when used in HPE ProLiant servers. HPE Standard Memory offers the best combination  of pricing, quality, reliability, and compatibility for HPE ProLiant servers—designed to help your business achieve powerful results with right-sized, affordable solutions.
In addition to describing these improvements, this white paper reviews the rules, best practices, and optimization strategies that should be used when installing HPE DDR4- 2933 on a HPE ProLiant Gen10 with 7002 and 7003 Processor or HPE DDR4-3200 memory on HPE ProLiant Gen10 Plus servers using AMD EPYC 7002 and 7003 processors.

POPULATING HPE DDR4 MEMORY IN HPE PROLIANT GEN10 PLUS SERVERS WITH AMD EPYC 7002 AND 7003 PROCESSORS

The high-level memory system architecture for HPE ProLiant Gen10 Plus servers using AMD EPYC 7002 and 7003 processors is different from that of the previous HPE  ProLiant servers. HPE ProLiant servers using AMD EPYC 7002 and 7003 processors integrates an IOD chip which includes 8 memory controllers to interface 8 memory channels per CPU and up to 32 DIMM slots in two-socket servers.

Population rules for HPE ProLiant Gen10 Plus servers with AMD EPYC 7002 and 7003 processors 
HPE ProLiant Gen10 Plus systems support a variety of flexible server memory configurations, enabling the system to be configured and run in any valid memory controller  configuration. For optimal performance and functionality, you should follow the rules when populating HPE ProLiant servers with HPE DDR4 memory. Violating these  rules may result in reduced memory capacity, performance, or error messages during boot. Table 1 summarizes the overall DIMM population rules for HPE ProLiant Gen10  Plus servers. TABLE 1. DIMM population rules for HPE ProLiant Gen10 Plus servers

CategoryPopulation guidelines
Processors and DIMM slotsInstall DIMMs only if the corresponding processor is installed. If only one processor is installed in a two-processor system, only half of the DIMM slots are available.
If a memory channel consists of more than one DIMM slot, the white DIMM slot will be located furthest from the CPU. White DIMM slots denote the first slot to be populated in a channel. For one DIMM per channel (DPC) populate white DIMM slots only.
When mixing DIMMs of different ranks on the same channel, place the DIMMs with the heaviest electrical load (highest number of ranks) in the white DIMM slot. Within a given channel, populate DIMMs from the heaviest electrical load (dual rank) to the lightest load (single rank).
If multiple CPUs are populated, split the DIMMs evenly across the CPUs and follow the corresponding CPU rule when populating DIMMs.
PerformanceTo maximize performance in a majority of the potential applications, it is recommended to balance the total memory capacity across all installed processors, channel pairs (A/B, C/D, E/F, and G/H). Load the channels similarly whenever possible to enable optimal interleaving. Populate all available channels first before installing two DIMMs in a channel. If the number of DIMMs does not spread evenly across the CPUs, populate as close to evenly as possible. Refer Figure 4.
DIMM types and capacitiesThe maximum memory capacity is a function of the number of DIMM slots on the platform: the largest DIMM capacity qualified on the platform and the number and model of qualified processors installed on the platform.
Do not mix RDIMMs and LRDIMMs in the same system. Do not mix 3DS DIMMs with non-3DS DIMMs in the same system. Do not mix x4 and x8 DRAM widths or 8 Gb and 16 Gb DRAM in the same system.
Unbuffered DIMMs (UDIMMs) are not supported.
DIMM speedThe maximum memory speed is a function of the memory type, memory configuration, and processor model. The server will select the highest common speed among all of the DIMMs present in the system.

There are several HPE ProLiant Gen10 Plus servers with different EPYC series. Table 2 shows EPYC series on each HPE ProLiant DL series Gen10 Plus servers. Table 3 shows EPYC series on each HPE ProLiant XL and HPE Apollo series Gen10 Plus servers.
TABLE 2. EPYC series on each HPE ProLiant DL series Gen10 Plus servers

HPE ProLiant DL series 1P ServersHPE ProLiant DL series 2P Servers
DL325 Gen10AMD EPYC 7002DL385 Gen10AMD EPYC 7002
DL325 Gen10 PlusAMD EPYC 7002DL385 Gen10 PlusAMD EPYC 7002
DL325 Gen10 Plus v2AMD EPYC 7003DL385 Gen10 Plus v2AMD EPYC 7003
DL345 Gen10 PlusAMD EPYC 7002 or 7003DL365 Gen10 PlusAMD EPYC 7002 or 7003

Technical white paper

TABLE 3. EPYC series on each HPE ProLiant XL and HPE Apollo series Gen10 Plus servers
HPE ProLiant XL and HPE Apollo series 2P Servers
XL225n Gen10 Plus and HPE Apollo AMD EPYC 7002 or 7003

DIMM connector location
In general, memory population order follows the same logic for all HPE ProLiant servers using AMD EPYC 7002 and 7003 processors although physical arrangement may vary from server to server.

Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - FIG

FIGURE 1. DIMM slot locations for 32-slot HPE ProLiant 2P Gen10 Plus two-socket configuration. The first DIMM slot for each channel on each processor are the white DIMM slots. Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - FIG 1

FIGURE 2. DIMM slot locations for 16-slot HPE ProLiant 1P Gen10 Plus one-socket servers. The first DIMM slot for each channel on the processor are the white DIMM slots.

HPE ProLiant XL225n Gen10 Plus and HPE Apollo Server Two Processor Configuration 1 slot per channelHewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - FIG 2

FIGURE 3. DIMM slot locations for 16-slot HPE ProLiant XL225n Gen10 Plus and HPE Apollo one-socket servers. Each DIMM slot corresponds with a channel.

DIMM population order
Figures 4, 5, and 6 shows the DIMM population order for HPE ProLiant Gen10 Plus servers with one AMD EPYC processor installed. There are 16 DIMM slots available  to be populated. For a given number of DIMMs, populate those DIMMs in the corresponding numbered DIMM slot(s) on the particular row based on server model as  shown in Figures 4, 5, 6, 7, and 8. The most optimal performance will be achieved if populating 4, 8, and 16 shown in green. If populating only 4 DIMMs, it is recommended  to use a processor with 32 or less cores. This is to prevent the memory bus to be heavily saturated by the greater amount of cores, which will impact performance.
HPE Server Memory should be installed as indicated based upon the total number of DIMMs being installed per CPU. For example, if two DIMMs are being installed per  CPU on a DL series 2P, they should be located in white DIMM slots numbered 16 and 14. If six DIMMs are being used per CPU, they should be installed in DIMM slots 16, 14, 12, 10, 3, and 1

HPE ProLiant DL series 2P Gen10  Plus—One Processor Configuration Homogeneous DIMM(s) Population Order
Number of DIMM(s) to populateProcessor 1
CH DCH CCH BCH ACH ECH FCH GCH H
114
21614
316143
4*161431
516141231
6**1614121031
6***161410731
716141210531
8161412107531
916141312107531
101615141312107531
1116151413121075431
12161514131210754321
1316151413121110754321
14161514131211109754321
151615141312111097654321
1616151413121110987654321

* Recommended only with processors that have 128 MB L3 cache or less.
** This is for EPYC 7002 only.
*** This is for EPYC 7003 only.
FIGURE 4. DIMM population order for HPE ProLiant DL series 2P Gen10 Plus servers with one processor installed (16 slots per processor).

HPE ProLiant DL series 1P Gen10  Plus—One Processor Configuration Homogeneous DIMM(s) Population Order
Number of DIMM(s) to populateProcessor 1
CH HCH GCH FCH ECH ACH BCH CCH D
13
231
31431
4*161431
51614531
6**16147531
6***161410731
71614127531
8161412107531
91614121075431
1016141210754321
111614131210754321
12161514131210754321
131615141312107654321
1416151413121087654321
151615141312111087654321
1616151413121110987654321

* Recommended only with processors that have 128 MB L3 cache or less.
** This is for EPYC 7002 only.
*** This is for EPYC 7003 only.
FIGURE 5. DIMM population order for HPE ProLiant DL series 1P Gen10 Plus servers with one processor installed (16 slots per processor).

HPE XL225n Gen10 Plus—Two Processor Configuration Homogeneous DIMM(s) Population Order
Number of DIMM(s) to populateProcessor 1
CH HCH GCH FCH ECH ACH BCH CCH D
12
221
3721
48721
587321
6*874321
6**875421
78764321
887654321

* This is for EPYC 7002 only.
** This is for EPYC 7003 only.
FIGURE 6. DIMM population order for HPE ProLiant XL225n Gen10 Plus and HPE Apollo servers with one processor installed (16 slots per processor).

Figures 7 and 8 show the DIMM population order for HPE ProLiant DL series 2P Gen10 Plus, and HPE ProLiant XL225n Gen10 Plus and HPE Apollo servers with two  processors installed. For a given number of DIMMs, populate those DIMMs in the corresponding numbered DIMM slot(s) on that row as shown in Figures 7 and 8.

HPE ProLiant DL series 2P Gen10 Plus—Two Processor Configuration Homogeneous DIMM(s) Population Order
Number of DIMM(s) to populateProcessor 2Processor 1
CH DCH CCH BCH ACH ECH FCH GCH HCH DCH CCH BCH ACH ECH FCH GCH H
114
21414
3141614
416141614
5161416143
61614316143
716143161431
8***161431161431
916143116141231
101614123116141231
11161412311614121031
12*16141210311614121031
12**161410731161410731
13161412103116141210531
141614121053116141210531
1516141210531161412107531
16161412107531161412107531
1716141210753116141312107531
181614131210753116141312107531
19161413121075311615141312107531
2016151413121075311615141312107531
21161514131210753116151413121075431
221615141312107543116151413121075431
2316151413121075431161514131210754321
24161514131210754321161514131210754321
2516151413121075432116151413121110754321
261615141312111075432116151413121110754321
2716151413121110754321161514131211109754321
28161514131211109754321161514131211109754321
291615141312111097543211615141312111097654321
3016151413121110976543211615141312111097654321
31161514131211109765432116151413121110987654321
321615141312111098765432116151413121110987654321

* This is for EPYC 7002 only.
** This is for EPYC 7003 only.
*** Recommended only with processors that have 128 MB L3 cache or less.
FIGURE 7. DIMM population order for HPE ProLiant DL series 2P Gen10 Plus servers with two processors installed (16 slots per processor).

HPE XL225n Gen10 Plus—Two Processor Configuration Homogeneous DIMM(s) Population Order
Number of DIMM(s) to populateProcessor 2Processor 1
CH DCH CCH BCH ACH ECH FCH GCH HCH HCH GCH FCH ECH ACH BCH CCH D
12
272
3721
48721
587721
6872721
78728721
887218721
9872187321
108762187321
1187621874321
12*876521874321
12**875421875421
138765218764321
1487653218764321
15876532187654321
168765432187654321

* This is for EPYC 7002 only.
** This is for EPYC 7003 only.
FIGURE 8. DIMM population order for HPE ProLiant XL225n Gen10 Plus and HPE Apollo servers with two processors installed (8 slots per processor).

Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - FIG 3

FIGURE 9. Supported capacities on Gen10 Plus using the recommended populations with 8 DIMMs installed. Homogeneous Configuration.

Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - FIG 4

FIGURE 10. Supported capacities on Gen10 Plus using the recommended populations with 16 DIMMs Installed. Homogeneous Configuration. For additional homogeneous configurations based on capacity see Additional Homogeneous Configurations section. Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - FIG 5

FIGURE 11. Supported capacities on Gen10 Plus using the recommended populations with 16 DIMMs Installed. Heterogeneous Configuration. For additional heterogeneous configurations based on capacity see Additional Heterogeneous Configurations section.

NUMA NODES

HPE ProLiant servers with EPYC 7002 and 7003 processors may support the option of configuring the NUMA (Non-Uniform Memory Access) NPS (Nodes Per Socket)  you can have on the processor. This will depend on the exact model number of the processor. This setting will be available in the RBSU->Memory Options->NUMA  memory domains per socket (1, 2, 4). For most workloads, one NUMA node will give the most optimal performance and is the default.
Nodes Per Socket (1)
This setting will assign one NUMA node per processor.
This setting will be the default and in general will give the best performance for most workloads.
Supports: 8-Way, 4-Way, and 2-Way interleaving modes depending on DIMM population.
Nodes Per Socket (2)
This setting will assign two NUMA nodes per processor.
Supports: 4-Way and 2-Way interleaving modes depending on DIMM population.
Nodes Per Socket (4)
This setting will assign four NUMA nodes per processor.
Supports: 2-Way interleaving mode depending on DIMM population.

MEMORY INTERLEAVING

Memory interleaving is a technique used to maximize memory performance by spreading memory addresses evenly across memory devices.
Interleaved memory results in a contiguous memory region across multiple devices with sequential accesses using each memory device in turn, instead of using the same one  repeatedly. HPE encourages enabling interleaving for the most optimal and deterministic performance.
The result is higher memory throughput due to the reduced wait times for memory banks to become available for desired operations between reads and writes.
Memory interleaving options include:

8-Way interleaving
When configured correctly, sequential reads will be interleaved across all memory channels (A/B/C/D/E/F/G/H). Channel bandwidth will be accumulated across the  interleaved channels. This is the optimal setting for NPS (1) configuration.
Can only be used with NPS (1) NUMA Node option.
6-Way interleaving 
When configured correctly, sequential reads will be interleaved across all memory channels (A/C/D/E/F/G/H). Channel bandwidth will be accumulated across the interleaved channels. This is the optimal setting for NPS (1) configuration.
Can only be used with NPS (1) NUMA Node option.
4-Way interleaving
When configured correctly, sequential reads will be interleaved across four memory channels: Channels (C/D/G/H) or Channels (A/B/E/F).
Channel bandwidth will be accumulated across the interleaved channels. This is the optimal setting for NPS (2) configuration.
Can be used with NPS (1) or NPS (2) NUMA Node options.
2-Way interleaving
When configured correctly, sequential reads will be interleaved across channel pairs; Channels (C/D), (G/H), (A/B), or (E/F). Channel bandwidth will be accumulated across  the interleaved channels. This is the only interleaving mode for a NPS (4) configuration.
Can be used with NPS (1), NPS (2), or NPS (4) NUMA Node options.

NOTE
When interleaving is enabled, the system will automatically attempt to do an 8-Way interleave first followed by 4-Way and 2-Way depending on the memory population and NPS selected.

Disabling memory interleaving
This option is available from the Advanced Power Management menu in the RBSU Memory Options menu if needed. HPE defaults to having interleaving enabled as this will  provide the best performance for most workloads. Disabling memory interleaving may decrease the overall memory performance.

Mixed DIMM configurations
LRDIMM and RDIMM should not be mixed (as per specification). 3DS and LRDIMM should also not be mixed as timings for these are significantly different, hence the  operation is reduced to slowest common timing.
No mixing of x4 and x8 memory, as it will cause slower memory and system operation.
No mixing of 8 Gb and 16 Gb DRAM, as it will cause slower memory and system operation.
On HPE servers based on AMD processors, mixing of DIMM capacities is supported as long as mixing rules are followed and the memory channels have identical memory  capacities to get the most performance. See Additional Heterogeneous Configurations.
Table 4 shows mixed DIMM configuration based on rules.
TABLE 4. Mixed population guidelines for HPE SmartMemory DIMMs

P/NDescriptionP07638- B21P07640- B21P07642- B21P07646- B21P38454- B21P07644- B21P07650- B21P07652- B21P07654- B21
8 GB 1Rx816 GB 1Rx416 GB 2Rx832 GB 2Rx432 GB 1Rx432 GB 2Rx864 GB 2Rx4128 GB 4Rx4256 GB 8Rx4 3DS
8 Gb8 Gb8 Gb8 Gb16 Gb16 Gb16 Gb16 Gb16 Gb
3200 MT/s3200 MT/s3200 MT/s3200 MT/s3200 MT/s3200 MT/s3200 MT/s3200 MT/s3200 MT/s
RDIMMRDIMMRDIMMRDIMMRDIMMRDIMMRDIMMLRDIMMLRDIMM
P07638-B21HPE 8GB 1Rx8 PC4-3200AA-R Smart KitYesYes
P07640-B21HPE 16GB 1Rx4 PC4-3200AA-R Smart KitYesYes
P07642-B21 

HPE 16GB 2Rx8 PC4-3200AA-R Smart Kit

YesYes
P07646-B21HPE 32GB 2Rx4 PC4-3200AA-R Smart KitYesYes
P38454-B21HPE 32GB 1Rx4 PC4- 3200AA-R Memory KitYesYes
P07644-B21HPE 32GB 2Rx8 PC4-3200AA-R Smart KitYes
P07650-B21HPE 64GB 2Rx4 PC4-3200AA-R Smart KitYesYes
P07652-B21HPE 128GB 4Rx4 PC4- 3200AA-L SmartYes
P07654-B21HPE 256GB 8Rx4 PC4- 3200AA-L 3DS SmartYes

CONCLUSION

HPE Smart Memory for HPE ProLiant Gen10 Plus AMD-based servers offers greater memory performance than ever before. The HPE DDR4-2933 and DDR4-3200  Smart Memory for HPE ProLiant Gen10 Plus servers that use the AMD EPYC 7002 and 7003 Processor delivers increased memory throughput and lower latencies. HPE  Smart Memory also provides extended performance in many configurations by operating at higher speeds compared to third-party memory. Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - FIG 6Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - FIG 7

Resources General
HPE servers technical white papers library
Memory
HPE Server Memory
HPE Server Memory Configurator
HPE Smart Memory whiteboard video
Memory speed tables for HPE Gen10 Plus servers using AMD EPYC 7002 and 7003-series processors
LEARN MORE AT
hpe.com/info/memory

Make the right purchase decision.
Contact our presales specialists.Hewlett Packard Enterprise GEN10 Servers with AMD EPYC 7002 and 7003 Series Processors - ICON

© Copyright 2017–2021 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett  Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be  construed as constituting an additional warranty.
Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.
AMD is a trademark of Advanced Micro Devices, Inc. All third-party marks are property of their respective owners.
a00038346ENW, May 2021, Rev. 6

References

Documents / Resouces

Download manual
Here you can download full pdf version of manual, it may contain additional safety instructions, warranty information, FCC rules, etc.


Related Manuals