- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Network virtual memory
Open Collections
UBC Theses and Dissertations
UBC Theses and Dissertations
Network virtual memory Ong, Joon Suan
Abstract
User-mode access, zero-copy transfer, and sender-managed communication have emerged as essential for improving communication performance in workstation and PC clusters. The goal of these techniques is to provide application-level DAAA to remote memory. Achieving this goal is difficult, however, because the network interface accesses physical rather than virtual memory. As a result, previous systems have confined source and destination data to pages in pinned physical memory. Unfortunately, this approach increases application complexity and reduces memory-management effectiveness. This thesis describes the design and implementation of NetVM, which is a network interface that supports user-mode access, zero-copy transfer and sender-managed communication without pinning source or destination memory. To do this, the network interface maintains a shadow page table, which the host operating system updates whenever it maps or unmaps a page in host memory. The network interface uses this table to briefly lock and translate the virtual address of a page when it accesses that page for DAAA transfer. The operating system is prevented from replacing a page in the short interval that the network interface has locked that page. If a destination page is not resident in memory, the network interface redirects the data to an intermediate system buffer, which the operating system uses to complete the transfer with a single host-to-host memory copy after fetching in the required page. A credit-based flow-control scheme prevents the system buffer from overflowing. Application-level DAAA transfers only data. To support control transfers, NetVM implements a counter-based notification mechanism for applications to issue and detect notifications. The sending application increments an event counter by specifying its identifier in an RDAAA write operation. The receiving application detects this event by busy waiting, block waiting or triggering a user-defined handler whenever the notifying write completes. This range of detection mechanisms allows the application to decide appropriate tradeoffs between reducing signaling latency and reducing processor overhead. NetVM enforces ordered notifications over an out-oforder delivery network by using a sequence window. NetVM supports efficient mutual-exclusion, wait-queue and semaphore synchronization implementations. It augments the network interface with atomic operation primitives, which have low overhead, to provide MCS-lock-inspired scalable and efficient high-level synchronization for applications. As a result, these operations require lower latency and fewer network transactions to complete compared with the traditional implementations. The NetVM prototype is implemented in firmware for the Myrinet LANai-9.2 and integrated with the FreeBSD 4.6 virtual memory system. NetVM's memory-management overhead is low; it adds only less than 5.0% write latency compared to a static pinning approach and has a lower pinning cost compared to a dynamic pinning approach that has up to 94.5% hit rate in the pinnedpage cache. Minimum write latency is 5.56us and maximum throughput is 155.46MB/s, which is 97.2% of the link bandwidth. Transferring control through notification adds between 2.96us and 17.49us to the write operation, depending on the detection mechanism used. Compared to standard low-level atomic operations, NetVM adds only up to 18.2% and 12.6% to application latencies for high-level wait-queue and counting-semaphore operations respectively.
Item Metadata
Title |
Network virtual memory
|
Creator | |
Publisher |
University of British Columbia
|
Date Issued |
2003
|
Description |
User-mode access, zero-copy transfer, and sender-managed communication have emerged as
essential for improving communication performance in workstation and PC clusters. The goal of
these techniques is to provide application-level DAAA to remote memory. Achieving this goal is
difficult, however, because the network interface accesses physical rather than virtual memory.
As a result, previous systems have confined source and destination data to pages in pinned
physical memory. Unfortunately, this approach increases application complexity and reduces
memory-management effectiveness.
This thesis describes the design and implementation of NetVM, which is a network interface
that supports user-mode access, zero-copy transfer and sender-managed communication without
pinning source or destination memory. To do this, the network interface maintains a
shadow page table, which the host operating system updates whenever it maps or unmaps a
page in host memory. The network interface uses this table to briefly lock and translate the
virtual address of a page when it accesses that page for DAAA transfer. The operating system is
prevented from replacing a page in the short interval that the network interface has locked
that page. If a destination page is not resident in memory, the network interface redirects the
data to an intermediate system buffer, which the operating system uses to complete the transfer
with a single host-to-host memory copy after fetching in the required page. A credit-based
flow-control scheme prevents the system buffer from overflowing.
Application-level DAAA transfers only data. To support control transfers, NetVM implements a
counter-based notification mechanism for applications to issue and detect notifications. The
sending application increments an event counter by specifying its identifier in an RDAAA write
operation. The receiving application detects this event by busy waiting, block waiting or triggering
a user-defined handler whenever the notifying write completes. This range of detection
mechanisms allows the application to decide appropriate tradeoffs between reducing signaling
latency and reducing processor overhead. NetVM enforces ordered notifications over an out-oforder
delivery network by using a sequence window.
NetVM supports efficient mutual-exclusion, wait-queue and semaphore synchronization implementations.
It augments the network interface with atomic operation primitives, which have
low overhead, to provide MCS-lock-inspired scalable and efficient high-level synchronization for
applications. As a result, these operations require lower latency and fewer network transactions
to complete compared with the traditional implementations.
The NetVM prototype is implemented in firmware for the Myrinet LANai-9.2 and integrated with
the FreeBSD 4.6 virtual memory system. NetVM's memory-management overhead is low; it adds
only less than 5.0% write latency compared to a static pinning approach and has a lower pinning
cost compared to a dynamic pinning approach that has up to 94.5% hit rate in the pinnedpage
cache. Minimum write latency is 5.56us and maximum throughput is 155.46MB/s, which is
97.2% of the link bandwidth. Transferring control through notification adds between 2.96us and
17.49us to the write operation, depending on the detection mechanism used. Compared to
standard low-level atomic operations, NetVM adds only up to 18.2% and 12.6% to application
latencies for high-level wait-queue and counting-semaphore operations respectively.
|
Extent |
8386000 bytes
|
Genre | |
Type | |
File Format |
application/pdf
|
Language |
eng
|
Date Available |
2009-11-13
|
Provider |
Vancouver : University of British Columbia Library
|
Rights |
For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
|
DOI |
10.14288/1.0051656
|
URI | |
Degree | |
Program | |
Affiliation | |
Degree Grantor |
University of British Columbia
|
Graduation Date |
2003-11
|
Campus | |
Scholarly Level |
Graduate
|
Aggregated Source Repository |
DSpace
|
Item Media
Item Citations and Data
Rights
For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.