форумы  помощь  поиск  регистрация  майллист  вход/выход  слежка

Вариант для распечатки		Пред. тема | След. тема
Форумы OpenNET: Виртуальная конференция (Public)
Изначальное сообщение		[ Отслеживать ]

"buffers 0k в топе: дисковый кэш не используется?"

Сообщение от davojan (ok) on 28-Июн-08, 11:20

Наблюдаю вот такую неприятную картинку в топе: top - 11:02:29 up 31 days, 11:37,  2 users,  load average: 0.46, 0.63, 0.60 Tasks: 178 total,   1 running, 177 sleeping,   0 stopped,   0 zombie Cpu0  :  4.9%us,  3.9%sy,  0.0%ni, 63.7%id, 27.5%wa,  0.0%hi,  0.0%si,  0.0%st Cpu1  :  1.0%us,  0.0%sy,  0.0%ni, 73.3%id, 25.7%wa,  0.0%hi,  0.0%si,  0.0%st Cpu2  :  4.9%us,  1.9%sy,  0.0%ni, 82.5%id, 10.7%wa,  0.0%hi,  0.0%si,  0.0%st Cpu3  :  2.9%us,  0.0%sy,  0.0%ni, 62.9%id, 34.3%wa,  0.0%hi,  0.0%si,  0.0%st Mem:    903556k total,   859436k used,    44120k free,        0k buffers Swap:   987988k total,     1428k used,   986560k free,   502412k cached При больших нагрузках практически всё процессорное время уходит в wa (я так понимаю, это ожилание ввода/вывода), буфера остаются нулевыми, даже если выключить все сервисы, жрущие память. На сервере крутится PostgreSQL. Linux 2.6.24-gentoo-r8 #1 SMP Tue May 27 16:46:06 MSD 2008 i686 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux Диски: 2 одинаковых sata-диска на dmraid'е (RAID 1). ФС: xfs Postgres оттюнен как следует, по нему вопросов нет. Насколько я понимаю, система работает без дискового кэша (или, может быть, сетевого?). Это смерти подобно. Облазил весь конфиг ядра и гугл - ничего не нашёл. Help! Заранее спасибо.

Высказать мнение | Ответить | Правка | Cообщить модератору

Сообщения по теме

[Сортировка по времени | RSS]

1. "buffers 0k в топе: дисковый кэш не используется?"

Сообщение от PavelR (??) on 28-Июн-08, 13:30

> >Postgres оттюнен как следует, по нему вопросов нет. Расскажите пожалуйста, каким образом осуществлен тюнинг субд-ы и что получилось в результате в конфиге ?

Высказать мнение | Ответить | Правка | Наверх | Cообщить модератору

	2. "buffers 0k в топе: дисковый кэш не используется?"
	Сообщение от davojan (ok) on 28-Июн-08, 16:15
	> >> >>Postgres оттюнен как следует, по нему вопросов нет. > >Расскажите пожалуйста, каким образом осуществлен тюнинг субд-ы и что получилось в результате >в конфиге ? show all;               name               |                 setting                  |                                         description ---------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------- add_missing_from                | off                                      | Automatically adds missing table references to FROM clauses. allow_system_table_mods         | off                                      | Allows modifications of the structure of system tables. archive_command                 |                                          | WAL archiving command. archive_timeout                 | 0                                        | Forces a switch to the next xlog file if a new file has not been started within N seconds. array_nulls                     | on                                       | Enable input of NULL elements in arrays. authentication_timeout          | 1min                                     | Sets the maximum time in seconds to complete client authentication. autovacuum                      | on                                       | Starts the autovacuum subprocess. autovacuum_analyze_scale_factor | 0.1                                      | Number of tuple inserts, updates or deletes prior to analyze as a fraction of reltuples. autovacuum_analyze_threshold    | 250                                      | Minimum number of tuple inserts, updates or deletes prior to analyze. autovacuum_freeze_max_age       | 200000000                                | Age at which to autovacuum a table to prevent transaction ID wraparound. autovacuum_naptime              | 2min                                     | Time to sleep between autovacuum runs, in seconds. autovacuum_vacuum_cost_delay    | -1                                       | Vacuum cost delay in milliseconds, for autovacuum. autovacuum_vacuum_cost_limit    | -1                                       | Vacuum cost amount available before napping, for autovacuum. autovacuum_vacuum_scale_factor  | 0.2                                      | Number of tuple updates or deletes prior to vacuum as a fraction of reltuples. autovacuum_vacuum_threshold     | 500                                      | Minimum number of tuple updates or deletes prior to vacuum. backslash_quote                 | safe_encoding                            | Sets whether "\'" is allowed in string literals. bgwriter_all_maxpages           | 5                                        | Background writer maximum number of all pages to flush per round bgwriter_all_percent            | 0.333                                    | Background writer percentage of all buffers to flush per round bgwriter_delay                  | 200ms                                    | Background writer sleep time between rounds in milliseconds bgwriter_lru_maxpages           | 5                                        | Background writer maximum number of LRU pages to flush per round bgwriter_lru_percent            | 1                                        | Background writer percentage of LRU buffers to flush per round block_size                      | 8192                                     | Shows size of a disk block bonjour_name                    |                                          | Sets the Bonjour broadcast service name. check_function_bodies           | on                                       | Check function bodies during CREATE FUNCTION. checkpoint_segments             | 3                                        | Sets the maximum distance in log segments between automatic WAL checkpoints. checkpoint_timeout              | 5min                                     | Sets the maximum time in seconds between automatic WAL checkpoints. checkpoint_warning              | 30s                                      | Logs if filling of checkpoint segments happens more frequently than this (in seconds). client_encoding                 | WIN1251                                  | Sets the client's character set encoding. client_min_messages             | notice                                   | Sets the message levels that are sent to the client. commit_delay                    | 0                                        | Sets the delay in microseconds between transaction commit and flushing WAL to disk. commit_siblings                 | 5                                        | Sets the minimum concurrent open transactions before performing commit_delay. config_file                     | /var/lib/postgresql/data/postgresql.conf | Sets the server's main configuration file. constraint_exclusion            | off                                      | Enables the planner to use constraints to optimize queries. cpu_index_tuple_cost            | 0.005                                    | Sets the planner's estimate of the cost of processing each index entry during an index scan. cpu_operator_cost               | 0.0025                                   | Sets the planner's estimate of the cost of processing each operator or function call. cpu_tuple_cost                  | 0.01                                     | Sets the planner's estimate of the cost of processing each tuple (row). custom_variable_classes         |                                          | Sets the list of known custom variable classes. data_directory                  | /var/lib/postgresql/data                 | Sets the server's data directory. DateStyle                       | ISO, MDY                                 | Sets the display format for date and time values. db_user_namespace               | off                                      | Enables per-database user names. deadlock_timeout                | 1s                                       | The time in milliseconds to wait on lock before checking for deadlock. debug_assertions                | off                                      | Turns on various assertion checks. debug_pretty_print              | off                                      | Indents parse and plan tree displays. debug_print_parse               | off                                      | Prints the parse tree to the server log. debug_print_plan                | off                                      | Prints the execution plan to server log. debug_print_rewritten           | off                                      | Prints the parse tree after rewriting to server log. default_statistics_target       | 10                                       | Sets the default statistics target. default_tablespace              |                                          | Sets the default tablespace to create tables and indexes in. default_transaction_isolation   | read committed                           | Sets the transaction isolation level of each new transaction. default_transaction_read_only   | off                                      | Sets the default read-only status of new transactions. default_with_oids               | off                                      | Create new tables with OIDs by default. dynamic_library_path            | $libdir                                  | Sets the path for dynamically loadable modules. effective_cache_size            | 128MB                                    | Sets the planner's assumption about size of the disk cache. enable_bitmapscan               | on                                       | Enables the planner's use of bitmap-scan plans. enable_hashagg                  | on                                       | Enables the planner's use of hashed aggregation plans. enable_hashjoin                 | on                                       | Enables the planner's use of hash join plans. enable_indexscan                | on                                       | Enables the planner's use of index-scan plans. enable_mergejoin                | on                                       | Enables the planner's use of merge join plans. enable_nestloop                 | on                                       | Enables the planner's use of nested-loop join plans. enable_seqscan                  | on                                       | Enables the planner's use of sequential-scan plans. enable_sort                     | on                                       | Enables the planner's use of explicit sort steps. enable_tidscan                  | on                                       | Enables the planner's use of TID scan plans. escape_string_warning           | on                                       | Warn about backslash escapes in ordinary string literals. explain_pretty_print            | on                                       | Uses the indented output format for EXPLAIN VERBOSE. external_pid_file               |                                          | Writes the postmaster PID to the specified file. extra_float_digits              | 0                                        | Sets the number of digits displayed for floating-point values. from_collapse_limit             | 8                                        | Sets the FROM-list size beyond which subqueries are not collapsed. fsync                           | on                                       | Forces synchronization of updates to disk. full_page_writes                | on                                       | Writes full pages to WAL when first modified after a checkpoint. geqo                            | on                                       | Enables genetic query optimization. geqo_effort                     | 5                                        | GEQO: effort is used to set the default for other GEQO parameters. geqo_generations                | 0                                        | GEQO: number of iterations of the algorithm. geqo_pool_size                  | 0                                        | GEQO: number of individuals in the population. geqo_selection_bias             | 2                                        | GEQO: selective pressure within the population. geqo_threshold                  | 12                                       | Sets the threshold of FROM items beyond which GEQO is used. gin_fuzzy_search_limit          | 0                                        | Sets the maximum allowed result for exact search by GIN. hba_file                        | /var/lib/postgresql/data/pg_hba.conf     | Sets the server's "hba" configuration file ident_file                      | /var/lib/postgresql/data/pg_ident.conf   | Sets the server's "ident" configuration file ignore_system_indexes           | off                                      | Disables reading from system indexes. integer_datetimes               | off                                      | Datetimes are integer based. join_collapse_limit             | 8                                        | Sets the FROM-list size beyond which JOIN constructs are not flattened. krb_caseins_users               | off                                      | Sets whether Kerberos user names should be treated as case-insensitive. krb_server_hostname             |                                          | Sets the hostname of the Kerberos server. krb_server_keyfile              |                                          | Sets the location of the Kerberos server key file. krb_srvname                     | postgres                                 | Sets the name of the Kerberos service. lc_collate                      | ru_RU.CP1251                             | Shows the collation order locale. lc_ctype                        | ru_RU.CP1251                             | Shows the character classification and case conversion locale. lc_messages                     | ru_RU.CP1251                             | Sets the language in which messages are displayed. lc_monetary                     | ru_RU.CP1251                             | Sets the locale for formatting monetary amounts. lc_numeric                      | ru_RU.CP1251                             | Sets the locale for formatting numbers. lc_time                         | ru_RU.CP1251                             | Sets the locale for formatting date and time values. listen_addresses                | localhost                                | Sets the host name or IP address(es) to listen to. local_preload_libraries         |                                          | Lists shared libraries to preload into each backend. log_connections                 | off                                      | Logs each successful connection. log_destination                 | stderr                                   | Sets the destination for server log output. log_directory                   | pg_log                                   | Sets the destination directory for log files. log_disconnections              | off                                      | Logs end of a session, including duration. log_duration                    | off                                      | Logs the duration of each completed SQL statement. log_error_verbosity             | default                                  | Sets the verbosity of logged messages. log_executor_stats              | off                                      | Writes executor performance statistics to the server log. log_filename                    | postgresql-%Y-%m-%d_%H%M.log             | Sets the file name pattern for log files. log_hostname                    | off                                      | Logs the host name in the connection logs. log_line_prefix                 |                                          | Controls information prefixed to each log line log_min_duration_statement      | 250ms                                    | Sets the minimum execution time in milliseconds above which statements will be logged. log_min_error_statement         | error                                    | Causes all statements generating error at or above this level to be logged. log_min_messages                | notice                                   | Sets the message levels that are logged. log_parser_stats                | off                                      | Writes parser performance statistics to the server log. log_planner_stats               | off                                      | Writes planner performance statistics to the server log. log_rotation_age                | 1d                                       | Automatic log file rotation will occur after N minutes log_rotation_size               | 10MB                                     | Automatic log file rotation will occur after N kilobytes log_statement                   | none                                     | Sets the type of statements logged. log_statement_stats             | off                                      | Writes cumulative performance statistics to the server log. log_truncate_on_rotation        | off                                      | Truncate existing log files of same name during log rotation. maintenance_work_mem            | 64MB                                     | Sets the maximum memory to be used for maintenance operations. max_connections                 | 400                                      | Sets the maximum number of concurrent connections. max_files_per_process           | 1000                                     | Sets the maximum number of simultaneously open files for each server process. max_fsm_pages                   | 1128000                                  | Sets the maximum number of disk pages for which free space is tracked. max_fsm_relations               | 70000                                    | Sets the maximum number of tables and indexes for which free space is tracked. max_function_args               | 100                                      | Shows the maximum number of function arguments. max_identifier_length           | 63                                       | Shows the maximum identifier length max_index_keys                  | 32                                       | Shows the maximum number of index keys. max_locks_per_transaction       | 64                                       | Sets the maximum number of locks per transaction. max_prepared_transactions       | 5                                        | Sets the maximum number of simultaneously prepared transactions. max_stack_depth                 | 2MB                                      | Sets the maximum stack depth, in kilobytes. password_encryption             | on                                       | Encrypt passwords. port                            | 5432                                     | Sets the TCP port the server listens on. post_auth_delay                 | 0                                        | Waits N seconds on connection startup after authentication. pre_auth_delay                  | 0                                        | no description available random_page_cost                | 4                                        | Sets the planner's estimate of the cost of a nonsequentially fetched disk page. redirect_stderr                 | on                                       | Start a subprocess to capture stderr output into log files. regex_flavor                    | advanced                                 | Sets the regular expression "flavor". search_path                     | "$user",public                           | Sets the schema search order for names that are not schema-qualified. seq_page_cost                   | 1                                        | Sets the planner's estimate of the cost of a sequentially fetched disk page. server_encoding                 | WIN1251                                  | Sets the server (database) character set encoding. server_version                  | 8.2.7                                    | Shows the server version. server_version_num              | 80207                                    | Shows the server version as an integer. shared_buffers                  | 6400kB                                   | Sets the number of shared memory buffers used by the server. shared_preload_libraries        |                                          | Lists shared libraries to preload into server. silent_mode                     | on                                       | Runs the server silently. sql_inheritance                 | on                                       | Causes subtables to be included by default in various commands. ssl                             | off                                      | Enables SSL connections. standard_conforming_strings     | off                                      | '...' strings treat backslashes literally. statement_timeout               | 0                                        | Sets the maximum allowed duration (in milliseconds) of any statement. stats_block_level               | on                                       | Collects block-level statistics on database activity. stats_command_string            | on                                       | Collects information about executing commands. stats_reset_on_server_start     | off                                      | Zeroes collected statistics on server restart. stats_row_level                 | on                                       | Collects row-level statistics on database activity. stats_start_collector           | on                                       | Starts the server statistics-collection subprocess. superuser_reserved_connections  | 3                                        | Sets the number of connection slots reserved for superusers. syslog_facility                 | LOCAL0                                   | Sets the syslog "facility" to be used when syslog enabled. syslog_ident                    | postgres                                 | Sets the program name used to identify PostgreSQL messages in syslog. tcp_keepalives_count            | 0                                        | Maximum number of TCP keepalive retransmits. tcp_keepalives_idle             | 0                                        | Seconds between issuing TCP keepalives. tcp_keepalives_interval         | 0                                        | Seconds between TCP keepalive retransmits. temp_buffers                    | 1024                                     | Sets the maximum number of temporary buffers used by each session. TimeZone                        | W-SU                                     | Sets the time zone for displaying and interpreting time stamps. timezone_abbreviations          | Default                                  | Selects a file of time zone abbreviations trace_notify                    | off                                      | Generates debugging output for LISTEN and NOTIFY. trace_sort                      | off                                      | Emit information about resource usage in sorting. transaction_isolation           | read committed                           | Sets the current transaction's isolation level. transaction_read_only           | off                                      | Sets the current transaction's read-only status. transform_null_equals           | off                                      | Treats "expr=NULL" as "expr IS NULL". unix_socket_directory           |                                          | Sets the directory where the Unix-domain socket will be created. unix_socket_group               |                                          | Sets the owning group of the Unix-domain socket. unix_socket_permissions         | 511                                      | Sets the access permissions of the Unix-domain socket. update_process_title            | on                                       | Updates the process title to show the active SQL command. vacuum_cost_delay               | 0                                        | Vacuum cost delay in milliseconds. vacuum_cost_limit               | 200                                      | Vacuum cost amount available before napping. vacuum_cost_page_dirty          | 20                                       | Vacuum cost for a page dirtied by vacuum. vacuum_cost_page_hit            | 1                                        | Vacuum cost for a page found in the buffer cache. vacuum_cost_page_miss           | 10                                       | Vacuum cost for a page not found in the buffer cache. vacuum_freeze_min_age           | 100000000                                | Minimum age at which VACUUM should freeze a table row. wal_buffers                     | 64kB                                     | Sets the number of disk-page buffers in shared memory for WAL. wal_sync_method                 | fdatasync                                | Selects the method used for forcing WAL updates out to disk. work_mem                        | 2MB                                      | Sets the maximum memory to be used for query workspaces. zero_damaged_pages              | off                                      | Continues processing past damaged page headers. sysctl -a | grep shm kernel.shmmax = 167772160 kernel.shmall = 2097152 kernel.shmmni = 4096 ipcs ------ Сегменты совм. исп. памяти -------- ключ       shmid      владелец   права      байты      nattch     состояние 0x00989299 163840     postgres  600        26312704   55 ------ Массивы семафоров -------- ключ       semid      владелец   права      nsems 0x00989299 4259840    postgres  600        17 0x0098929a 4292609    postgres  600        17 0x0098929b 4325378    postgres  600        17 0x0098929c 4358147    postgres  600        17 0x0098929d 4390916    postgres  600        17 0x0098929e 4423685    postgres  600        17 0x0098929f 4456454    postgres  600        17 0x009892a0 4489223    postgres  600        17 0x009892a1 4521992    postgres  600        17 0x009892a2 4554761    postgres  600        17 0x009892a3 4587530    postgres  600        17 0x009892a4 4620299    postgres  600        17 0x009892a5 4653068    postgres  600        17 0x009892a6 4685837    postgres  600        17 0x009892a7 4718606    postgres  600        17 0x009892a8 4751375    postgres  600        17 0x009892a9 4784144    postgres  600        17 0x009892aa 4816913    postgres  600        17 0x009892ab 4849682    postgres  600        17 0x009892ac 4882451    postgres  600        17 0x009892ad 4915220    postgres  600        17 0x009892ae 4947989    postgres  600        17 0x009892af 4980758    postgres  600        17 0x009892b0 5013527    postgres  600        17 0x009892b1 5046296    postgres  600        17 0x009892b2 5079065    postgres  600        17 ------ Очереди сообщений -------- ключ       msqid      владелец   права      исп. байты   сообщения Единственное, что мне не нравится тут, это shared_buffers: show all показывает 6400kB, хотя в конфиге нарисовано shared_buffers = 8192 (это эквивалентно 64M), ipcs показывает вообще 26 метров (правда я не знаю, что означает эта цифра). Мне кажется, что бы там ни было, в топе не должно быть buffers 0k, ситуация не меняется, даже если я останавливаю практически все сервисы, включая postgresql.
	Высказать мнение | Ответить | Правка | Наверх | Cообщить модератору

3. "buffers 0k в топе: дисковый кэш не используется?"

Сообщение от Аноним (??) on 28-Июн-08, 16:36

>Наблюдаю вот такую неприятную картинку в топе... А что в  /proc/meminfo ?

Высказать мнение | Ответить | Правка | Наверх | Cообщить модератору

	5. "buffers 0k в топе: дисковый кэш не используется?"
	Сообщение от davojan (ok) on 29-Июн-08, 23:34
	MemTotal:      2072308 kB MemFree:        149248 kB Buffers:             0 kB Cached:        1581304 kB SwapCached:       1608 kB Active:         793664 kB Inactive:      1017156 kB HighTotal:     1178048 kB HighFree:        24436 kB LowTotal:       894260 kB LowFree:        124812 kB SwapTotal:      987988 kB SwapFree:       980120 kB Dirty:           24264 kB Writeback:           0 kB AnonPages:      227900 kB Mapped:         196968 kB Slab:            83952 kB SReclaimable:    59264 kB SUnreclaim:      24688 kB PageTables:       7864 kB NFS_Unstable:        0 kB Bounce:              0 kB CommitLimit:   2024140 kB Committed_AS:   719204 kB VmallocTotal:   114680 kB VmallocUsed:      5036 kB VmallocChunk:    92688 kB HugePages_Total:     0 HugePages_Free:      0 HugePages_Rsvd:      0 HugePages_Surp:      0 Hugepagesize:     4096 kB
	Высказать мнение | Ответить | Правка | Наверх | Cообщить модератору

4. "buffers 0k в топе: дисковый кэш не используется?"

Сообщение от davojan (ok) on 29-Июн-08, 01:26

Спасибо всем, кто попытался помочь. Проблема решилась. Не знаю, как буфера с этим связаны. Но в ядре не был включен high memory support (т.е. возможность использования более 1ГБ памяти), хотя памяти стоит 2 гига (можете смеяться :). После включения этой опции и перезагрузки буфера появились. Хотя опять же, совершенно связь не ясна, их не было, даже когда свободной памяти было до фига. С shared_buffers в PostgreSQL тоже разобрался. Кривое значение было задано в /etc/conf.d/postgresql и перекрывало правильное значение, которое было задано в postgresql.conf. В общем, проблема вроде бы решена, посмотрим в понедельник, будет ли легче под нагрузкой. Если кто-нибудь объяснит связь включённости high memory support и использования буферов, тому пряник :)

Высказать мнение | Ответить | Правка | Наверх | Cообщить модератору

6. "buffers 0k в топе: дисковый кэш не используется?"

Сообщение от davojan (ok) on 29-Июн-08, 23:36

Рано порадовался. Только что посмотрел - опять 0k buffers. Содержимое /proc/meminfo запостил чуть выше. Буду рад совету - куда копать?..

Высказать мнение | Ответить | Правка | Наверх | Cообщить модератору

7. "buffers 0k в топе: дисковый кэш не используется?"

Сообщение от davojan (ok) on 02-Июл-08, 12:29

Как выяснилось, buffers 0k объясняется тем, что у меня везде xfs - он использует кэши как-то по своему и они не отражаются в buffers. Однако меня продолжает беспокоить очень большой процент процессорного времени на wa в топе. Кто-нибудь может сказать, как выяснить, что у меня тормозит? Или так и должно быть?

Высказать мнение | Ответить | Правка | Наверх | Cообщить модератору

	8. "buffers 0k в топе: дисковый кэш не используется?"
	Сообщение от renton (ok) on 04-Авг-08, 13:45
	>Как выяснилось, buffers 0k объясняется тем, что у меня везде xfs - >он использует кэши как-то по своему и они не отражаются в >buffers. А это вы где прочитали?
	Высказать мнение | Ответить | Правка | Наверх | Cообщить модератору

Архив | Удалить	Индекс форумов | Темы | Пред. тема | След. тема
Оцените тред (1=ужас, 5=супер)? [ 1 | 2 | 3 | 4 | 5 ] [Рекомендовать для помещения в FAQ]