[Linux] Memory values shown by memory pressure handler logger are not useful

https://bugs.webkit.org/show_bug.cgi?id=164589

Reviewed by Michael Catanzaro.

We are currently using the VmSize field from /proc/self/status which is the virtual memory size of the process
and doesn't normally change even when the memory pressure handler manages to release memory. So, most of the
time we see that there's no changes in memory usage in the logs.
We should use the actual memory used by the process, memory that the process can release and then it's relevant
for the memory pressure handler. Using other fields from /proc/self/status we could do something like VmRSS -
(RssFile + RssShme), but there's also /proc/self/statm that provides the same information in a single. The main
different is that statm provides both resident and shared memory directly, but in number of pages, so we need to
multiply by the size of the page.
This patch adds a method to parse /proc/self/statm in its given file, because I plan to use this for the linux
memory sampler that is incorrectly parsing /proc/self/statm.

* platform/Linux.cmake: Add new files to compilation.
* platform/linux/CurrentProcessMemoryStatus.cpp: Added.
(WebCore::systemPageSize): Return the page size.
(WebCore::currentProcessMemoryStatus): Parse /proc/self/statm and fill the given ProcessMemoryStatus.
* platform/linux/CurrentProcessMemoryStatus.h: Added.
* platform/linux/MemoryPressureHandlerLinux.cpp:
(WebCore::MemoryPressureHandler::processMemoryUsage(): Helper function to return the memory used by the process
in bytes.
(WebCore::MemoryPressureHandler::ReliefLogger::platformMemoryUsage): Use processMemoryUsage().

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@208536 268f45cc-cd09-0410-ab3c-d52691b4dbfc

diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog
index 6444060..d849877 100644 (file)
--- a/Source/WebCore/ChangeLog
+++ b/Source/WebCore/ChangeLog
@@ -1,3 +1,31 @@
+2016-11-10  Carlos Garcia Campos  <cgarcia@igalia.com>
+
+        [Linux] Memory values shown by memory pressure handler logger are not useful
+        https://bugs.webkit.org/show_bug.cgi?id=164589
+
+        Reviewed by Michael Catanzaro.
+
+        We are currently using the VmSize field from /proc/self/status which is the virtual memory size of the process
+        and doesn't normally change even when the memory pressure handler manages to release memory. So, most of the
+        time we see that there's no changes in memory usage in the logs.
+        We should use the actual memory used by the process, memory that the process can release and then it's relevant
+        for the memory pressure handler. Using other fields from /proc/self/status we could do something like VmRSS -
+        (RssFile + RssShme), but there's also /proc/self/statm that provides the same information in a single. The main
+        different is that statm provides both resident and shared memory directly, but in number of pages, so we need to
+        multiply by the size of the page.
+        This patch adds a method to parse /proc/self/statm in its given file, because I plan to use this for the linux
+        memory sampler that is incorrectly parsing /proc/self/statm.
+
+        * platform/Linux.cmake: Add new files to compilation.
+        * platform/linux/CurrentProcessMemoryStatus.cpp: Added.
+        (WebCore::systemPageSize): Return the page size.
+        (WebCore::currentProcessMemoryStatus): Parse /proc/self/statm and fill the given ProcessMemoryStatus.
+        * platform/linux/CurrentProcessMemoryStatus.h: Added.
+        * platform/linux/MemoryPressureHandlerLinux.cpp:
+        (WebCore::MemoryPressureHandler::processMemoryUsage(): Helper function to return the memory used by the process
+        in bytes.
+        (WebCore::MemoryPressureHandler::ReliefLogger::platformMemoryUsage): Use processMemoryUsage().
+

 2016-10-14  Carlos Garcia Campos  <cgarcia@igalia.com>
 
         MemoryPressureHandler shouldn't know how to release WebCore memory
 2016-10-14  Carlos Garcia Campos  <cgarcia@igalia.com>
 
         MemoryPressureHandler shouldn't know how to release WebCore memory

diff --git a/Source/WebCore/platform/Linux.cmake b/Source/WebCore/platform/Linux.cmake
index f51b208..520ddcc 100644 (file)
--- a/Source/WebCore/platform/Linux.cmake
+++ b/Source/WebCore/platform/Linux.cmake
@@ -5,5 +5,7 @@ list(APPEND WebCore_INCLUDE_DIRECTORIES
 
 list(APPEND WebCore_SOURCES
     platform/gamepad/linux/GamepadDeviceLinux.cpp
 
 list(APPEND WebCore_SOURCES
     platform/gamepad/linux/GamepadDeviceLinux.cpp

+
+    platform/linux/CurrentProcessMemoryStatus.cpp

     platform/linux/MemoryPressureHandlerLinux.cpp
 )
     platform/linux/MemoryPressureHandlerLinux.cpp
 )

diff --git a/Source/WebCore/platform/linux/CurrentProcessMemoryStatus.cpp b/Source/WebCore/platform/linux/CurrentProcessMemoryStatus.cpp
new file mode 100644 (file)
index 0000000..36139ab
--- /dev/null
+++ b/Source/WebCore/platform/linux/CurrentProcessMemoryStatus.cpp
@@ -0,0 +1,77 @@
+/*
+ * Copyright (C) 2016 Igalia S.L.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "CurrentProcessMemoryStatus.h"
+
+#if OS(LINUX)
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+namespace WebCore {
+
+static inline size_t systemPageSize()
+{
+    static size_t pageSize = 0;
+    if (!pageSize)
+        pageSize = sysconf(_SC_PAGE_SIZE);
+    return pageSize;
+}
+
+void currentProcessMemoryStatus(ProcessMemoryStatus& memoryStatus)
+{
+    FILE* file = fopen("/proc/self/statm", "r");
+    if (!file)
+        return;
+
+    char buffer[128];
+    char* line = fgets(buffer, 128, file);
+    fclose(file);
+    if (!line)
+        return;
+
+    size_t pageSize = systemPageSize();
+    char* end = nullptr;
+    unsigned long long intValue = strtoull(line, &end, 10);
+    memoryStatus.size = intValue * pageSize;
+    intValue = strtoull(end, &end, 10);
+    memoryStatus.resident = intValue * pageSize;
+    intValue = strtoull(end, &end, 10);
+    memoryStatus.shared = intValue * pageSize;
+    intValue = strtoull(end, &end, 10);
+    memoryStatus.text = intValue * pageSize;
+    intValue = strtoull(end, &end, 10);
+    memoryStatus.lib = intValue * pageSize;
+    intValue = strtoull(end, &end, 10);
+    memoryStatus.data = intValue * pageSize;
+    intValue = strtoull(end, &end, 10);
+    memoryStatus.dt = intValue * pageSize;
+}
+
+} // namespace WebCore
+
+#endif // OS(LINUX)

diff --git a/Source/WebCore/platform/linux/CurrentProcessMemoryStatus.h b/Source/WebCore/platform/linux/CurrentProcessMemoryStatus.h
new file mode 100644 (file)
index 0000000..b6d822b
--- /dev/null
+++ b/Source/WebCore/platform/linux/CurrentProcessMemoryStatus.h
@@ -0,0 +1,46 @@
+/*
+ * Copyright (C) 2016 Igalia S.L.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if OS(LINUX)
+
+namespace WebCore {
+
+struct ProcessMemoryStatus {
+    size_t size { 0 };
+    size_t resident { 0 };
+    size_t shared { 0 };
+    size_t text { 0 };
+    size_t lib { 0 };
+    size_t data { 0 };
+    size_t dt { 0 };
+};
+
+void currentProcessMemoryStatus(ProcessMemoryStatus&);
+
+} // namespace WebCore
+
+#endif // OS(LINUX)

diff --git a/Source/WebCore/platform/linux/MemoryPressureHandlerLinux.cpp b/Source/WebCore/platform/linux/MemoryPressureHandlerLinux.cpp
index c2d4929..9a03832 100644 (file)
--- a/Source/WebCore/platform/linux/MemoryPressureHandlerLinux.cpp
+++ b/Source/WebCore/platform/linux/MemoryPressureHandlerLinux.cpp
@@ -29,6 +29,7 @@
 
 #if OS(LINUX)
 
 
 #if OS(LINUX)
 

+#include "CurrentProcessMemoryStatus.h"

 #include "Logging.h"
 
 #include <errno.h>
 #include "Logging.h"
 
 #include <errno.h>


@@ -59,7 +60,6 @@ static const unsigned s_holdOffMultiplier = 20;
 
 static const char* s_cgroupMemoryPressureLevel = "/sys/fs/cgroup/memory/memory.pressure_level";
 static const char* s_cgroupEventControl = "/sys/fs/cgroup/memory/cgroup.event_control";
 
 static const char* s_cgroupMemoryPressureLevel = "/sys/fs/cgroup/memory/memory.pressure_level";
 static const char* s_cgroupEventControl = "/sys/fs/cgroup/memory/cgroup.event_control";

-static const char* s_processStatus = "/proc/self/status";

 
 #if USE(GLIB)
 typedef struct {
 
 #if USE(GLIB)
 typedef struct {


@@ -157,27 +157,6 @@ void MemoryPressureHandler::EventFDPoller::readAndNotify() const
     m_notifyHandler();
 }
 
     m_notifyHandler();
 }
 

-static inline String nextToken(FILE* file)
-{
-    if (!file)
-        return String();
-
-    static const unsigned bufferSize = 128;
-    char buffer[bufferSize] = {0, };
-    unsigned index = 0;
-    while (index < bufferSize) {
-        int ch = fgetc(file);
-        if (ch == EOF || (isASCIISpace(ch) && index)) // Break on non-initial ASCII space.
-            break;
-        if (!isASCIISpace(ch)) {
-            buffer[index] = ch;
-            index++;
-        }
-    }
-
-    return String(buffer);
-}
-

 inline void MemoryPressureHandler::logErrorAndCloseFDs(const char* log)
 {
     if (log)
 inline void MemoryPressureHandler::logErrorAndCloseFDs(const char* log)
 {
     if (log)


@@ -293,6 +272,13 @@ void MemoryPressureHandler::holdOff(unsigned seconds)
     m_holdOffTimer.startOneShot(seconds);
 }
 
     m_holdOffTimer.startOneShot(seconds);
 }
 

+static size_t processMemoryUsage()
+{
+    ProcessMemoryStatus memoryStatus;
+    currentProcessMemoryStatus(memoryStatus);
+    return (memoryStatus.resident - memoryStatus.shared);
+}
+

 void MemoryPressureHandler::respondToMemoryPressure(Critical critical, Synchronous synchronous)
 {
     uninstall();
 void MemoryPressureHandler::respondToMemoryPressure(Critical critical, Synchronous synchronous)
 {
     uninstall();


@@ -313,22 +299,7 @@ void MemoryPressureHandler::platformReleaseMemory(Critical)
 
 size_t MemoryPressureHandler::ReliefLogger::platformMemoryUsage()
 {
 
 size_t MemoryPressureHandler::ReliefLogger::platformMemoryUsage()
 {

-    FILE* file = fopen(s_processStatus, "r");
-    if (!file)
-        return static_cast<size_t>(-1);
-
-    size_t vmSize = static_cast<size_t>(-1); // KB
-    String token = nextToken(file);
-    while (!token.isEmpty()) {
-        if (token == "VmSize:") {
-            vmSize = nextToken(file).toInt() * KB;
-            break;
-        }
-        token = nextToken(file);
-    }
-    fclose(file);
-
-    return vmSize;
+    return processMemoryUsage();

 }
 
 void MemoryPressureHandler::setMemoryPressureMonitorHandle(int fd)
 }
 
 void MemoryPressureHandler::setMemoryPressureMonitorHandle(int fd)