You can check the MySQL server’s timezone by the following command

SELECT @@global.time_zone; # check the global time zone 
SELECT @@session.time_zone; # check the current session timezone 
SELECT TIMEDIFF(NOW(), UTC_TIMESTAMP); # check the current time zone compared to UTC
SELECT UNIX_TIMESTAMP(); # check the current system timestamp
SELECT UNIX_TIMESTAMP(NOW());


SET time_zone = 'Europe/Helsinki';
SET time_zone = "+00:00";
SET @@session.time_zone = "+00:00"; 


 

Refer:

  1. Should MySQL have its timezone set to UTC?

Version: MySQL 5.7

Graphic describing InnoDB Group Replication, MySQL Router and MySQL Shell

  • create 3 sandbox instance:

after login to mysqljs:

dba.deploySandboxInstance(3310);
dba.deploySandboxInstance(3320);
dba.deploySandboxInstance(3330);

\connect root@localhost:3310

var cluster = dba.createCluster('DevCluster');


cluster.status(); 

cluster.addInstance('root@localhost:3320');
cluster.addInstance('root@localhost:3330');

cluster.status();



sudo mysqlrouter --bootstrap localhost:3310

mysqlsh --uri root@localhost:6446 --sql

select @@port;


 

explain SELECT * FROM `click_tripadvisor` where refid='W@OYAAoQK3oAAv-wD0AAAADF';
explain SELECT * FROM `click_tripadvisor` where provider_hotel_code='HOMRST';

set global slow_query_log=1; # open the slow_query_log option, to log the slow query into given file

show global variables like '%slow%';

select @@global.long_query_time; # default long query time

set global long_query_time = 0; # set 0 second to show the queery time

set global slow_query_log=0; # close the slow_query_log option

find / -name 'ip-10-200-107-82-slow.log'
watch ls -lh /usr/local/mariadb-10.0.36-linux-x86_64/data/ip-10-200-107-82-slow.log


# download pt-query-digest
wget percona.com/get/pt-query-digest
chmod u+x pt-query-digest

# analyze the slow log
./pt-query-digest /usr/local/mariadb-10.0.36-linux-x86_64/data/ip-10-200-107-82-slow.log > slow-query.log.out
yum install -y perl-libwww-perl perl-Time-HiRes


# analyze the slow log use the tcpdump
tcpdump -s 65535 -x -nn -q -tttt -i any -c 9999999 port 3306 | gzip -c > /tmp/tcpdump.txt.gz 
gunzip -c /tmp/tcpdump.txt.gz | ./pt-query-digest --type tcpdump > /tmp/digest_output.txt


# dump mysql data
mongodump --host localhost --port 27017 --db dacms --out mongo_bak/2018-11-16 -u mongodb -p pass --authenticationDatabase admin
mysqldump -uroot -p --routines --events --all-databases --force > all-database.sql

# PT 2.1
 pt-query-digest --processlist h=localhost --print --no-report --interval=0.01 > slow.log
# PT 2.2
 pt-query-digest --processlist h=localhost --interval=0.01 --output=slowlog > slow.log 

# filter the slow log by the given conditions
pt-query-digest --since '2015-04-10 00:00:00' --util '2015-04-10-23:59:59' /usr/local/mariadb-10.0.36-linux-x86_64/data/ip-10-200-107-82-slow.log > slowquery.log
pt-query-digest --filter '$event->{db} && $event->{db} =~ /`linkcenter-data`/' /usr/local/mariadb-10.0.36-linux-x86_64/data/ip-10-200-107-82-slow.log > employee-db-slowquery.log
pt-query-digest --filter '$event->{db} && $event->{db} =~ /market/' /usr/local/mariadb-10.0.36-linux-x86_64/data/ip-10-200-107-82-slow.log > market-db-slowquery.log
pt-query-digest --filter '$event->{user} && $event->{user} =~ /root/' /usr/local/mariadb-10.0.36-linux-x86_64/data/ip-10-200-107-82-slow.log > root-user-slowquery.log


# https://www.percona.com/blog/2011/12/29/identifying-the-load-with-the-help-of-pt-query-digest-and-percona-server/
# min_examined_row_limit: how many records query needs to analyze before it gets written to the query log
# log_queries_not_using_indexes: log ALL queries that didn't use an index
# log_output: FILE or TABLE. keep it set to FILE
# show_query_log_use_global_variable: it controls which variables will use global rather than session variables
# log_all_rate_limit: e.g. log_slow_rate_limit=10 means only log 1/10th of sessions regradless of long_query_time
# log_slow_rate_type: can be set to query in which case log_slow_rate_limit will limit by query, not by session
# slow_query_log_always_write_time: will log queries that take this long regardless of the rate limiter setting
# log_slow_slave_statements: replication statements are not logged by default
# log_slow_sp_statements: useful with stored routines
# 




# show the actual data size and index size
select engine, count(*) as TABLES, 
  concat(round(sum(TABLE_ROWS)/1000000,2), 'M') rows, 
  concat(round(sum(data_length)/(1024*1024*1024),2), 'G') DATA,
  concat(round(sum(index_length)/(1024*1024*1024),2), 'G') idx,
  concat(round(sum(data_length+index_length)/(1024*1024*1024),2), 'G') total_size,
  round(sum(index_length)/sum(data_length), 2) idxfrac
from information_schema.TABLES
where table_schema not in ('mysql', 'performance_schema', 'information_schema')
group by ENGINE
ORDER BY SUM(data_length + index_length) DESC LIMIT 10;

show variables like '%innodb_buffer_pool_size%';
show variables like '%packet%';

# catch teh actual innodb buffer read stream
mysqladmin -uroot -pderbysoft ext -ri1 | grep Innodb_buffer_pool_reads




# use mysqltunner to tune mysql configuration
wget http://mysqltuner.pl/ -O mysqltuner.pl
perl mysqltuner.pl --user root --pass derbysoft



 

check the original post at https://www.rathishkumar.in/2017/01/how-to-allocate-innodb-buffer-pool-size-in-mysql.html

The innodb_buffer_pool_size the most important variable for entire MySQL server. Last few years, you have seen the drastic improvements in the InnoDB storage engine features. After Oracle acquiring the MySQL, the importance of InnoDB storage engine grows multiple times over MyISAM. Even though MySQL supports multiple pluggable storage engines, one of the main reasons is its ACID compliance properties. Recent improvements in the MySQL 5.7 such as online innodb_buffer_pool_size configuration made it as a highly flexible and allow us to get the best performance without long downtime.

Initially, when I start to configure the innodb_buffer_pool_size variable on my test environment, like most of us, I googled as,

What is MySQL innodb_buffer_pool_size?
Do I need to change MySQL innodb_buffer_pool_size?
What are the recommendations for innodb_buffer_pool_size in MySQL?
What are the best practices for configuring innodb_buffer_pool_size in MySQL?
How to MySQL innodb_buffer_pool_size?
What is the best value for MySQL innodb_buffer_pool_size?
How to choose MySQL innodb_buffer_pool_size?
What is the optimal value for MySQL innodb_buffer_pool_size?
How large should be the MySQL innodb_buffer_pool_size?

The google results for the above keywords given me insights about MySQL innodb_buffer_pool_size. In my recent task, I created a new MySQL replication slave server on my production environment. During this process, I just implemented, whatever I learned from my testing experience.

The result was unbelievable, I was running MySQL 5.5 with 8 core CPU and 32 GB memory on a Linux platform. My backup file was around 320GB. The backup file took around 28 hours to restore on the MySQL 5.7 with default configuration on the similar server machine. Most of the tables are in InnoDB storage engine. After applying those InnoDB configurations, the restoration took just 6 hours to complete.

The dramatic change in the restoration time makes me dive deeper into the MySQL performance tuning. There are around 450 settings in MySQL 5.7, but I feel the most important variable setting is innodb_buffer_pool_size. This post is all about only MySQL innodb_buffer_pool_size. On my environment, most of the tables are in InnoDB, so this makes sense. It may be different in your case, still better understanding of this variable makes the life easier in most cases.

  • Why InnoDB buffer pool?

From the early stages of MySQL, the most widely used storage engine and default storage engine until MySQL 5.5 was MyISAM. MyISAM is using Operating System file cache to cache data that queries are reading over and over again. InnoDB handles caching itself, within the InnoDB buffer pool. So it is highly required to have enough InnoDB buffer pool space.

  • What is there inside MySQL InnoDB buffer pool?

MySQL InnoDB buffer pool contains the following things inside:

Data caching – InnoDB data pages.
Indices caching – index data.
Buffering data – Dirty pages – data which are modified in memory but not yet flushed (written) to a data disk.
Internal structures – InnoDB buffer pool additionally stores the internal structures such as Adaptive Hash Index, row level locks, etc.

The unit of the above-mentioned objects are calculated as – InnoDB pages. Each InnoDB pages is 16k in size.

  • How to choose the optimal InnoDB buffer pool size?

InnoDB buffer pool size is configured by the variable – innodb_buffer_pool_size. Before MySQL 5.7, this variable can be configured only at the offline. From MySQL 5.7 this can be configured online, does not require a server restart.

In an existing server, you can see the default value of innodb_buffer_pool_size by using the following command:

SHOW VARIABLES LIKE ‘%innodb_buffer_pool_size%’;

It is recommended that the buffer pool should be capable of holding entire database. But when we consider the practical situation it is not always possible.

Let me tell you an example, my server machine has around 32GB of RAM. My database size is around 300 GB. In this scenario, it is not possible to allocate 300 GB of RAM to innodb_buffer_poool_size variable.

  • How to deal with this situation?

When we cannot keep an entire database in memory, we should try to keep at least working data set in memory. In most cases, we are not going to process the data of an entire table, so the required data set should be in memory. So it is always better to allocate around 75% – 80% of the total available memory of server machine to innodb_buffer_pool_size.

  • How to set innodb_buffer_pool_size value?

This variable can be configured online from MySQL 5.7 using the following command:

SET GLOBAL innodb_buffer_pool_size = (value in bytes);

You should be very cautious to set the value for this variable at online. Because, when you restart the server, you will lose this configuration value. It is recommended to keep track of this changes and when you restart the server, you must set this value on the configuration file my.cnf.

Before MySQL 5.7 you have to set this configuration offline. You can set this value by editing my.cnf configuration file. Open my.cnf file and under the [mysqld] section, edit the following line.

innodb_buffer_pool_size = X G;

Now restart the MySQL server, and check the new configuration value by running

SHOW VARIABLES LIKE ‘%innodb_buffer_pool_size%’;
Example:
Online method:
SET GLOBAL innodb_buffer_pool_size = 26843545600;

Offline method:
innodb_buffer_pool = 26G

Note: Server restart required here.

  • Is innodb_buffer_pool_size is the reason for slow performance?

There is no doubt that innodb_buffer_pool_size is one of the most important variables for MySQL server. In the right scenario having a larger innodb_buffer_pool_size, will increase your database server performance dramatically. But is this always true?. No. We should have enough buffer pool, but it is not the only cause for slow performance. You can verify this by analyzing the performance of the InnoDB buffer pool.

The InnoDB buffer pool performance can be calculated by using the following formula:

Performance = innodb_buffer_pool_reads / innodb_buffer_pool_read_requests * 100

innodb_buffer_pool_reads: It indicates the number of requests that cannot be satisfied with InnoDB buffer pool. Need to read from the disk.

innodb_buffer_pool_read_requests: It indicates the number of requests of logical reads from memory.

For example on my server, let me check the InnoDB buffer pool performance:

innodb_buffer_pool_reads = 91661
innodb_buffer_pool_read_requests = 4029033624

Performance = 91661 / 4029033624 * 100

InnoDB Performance = 0.0022750120389663. This means the InnoDB can satisfy most of the requests from Buffer pool itself. Very minimal percentage of reads are done from the disk. So there is no need to increase the innodb_buffer_pool_size value.

So the logic is if your InnoDB buffer pool is able to satisfy all your query requests, then you have enough innodb_buffer_pool_size. If it reads from disk (the working data set is not available in buffer pool memory) then you should increase the innodb_buffer_pool_size.

When the performance of the InnoDB buffer pool maximum, it should not be the reason for the slow performance of the database server, in this case, you can check for other configurations.

  • When to decrease the innodb_buffer_pool_size?

Having larger innodb_buffer_pool_size is not always mandatory, but still, it depends on the database size and working data set.If it is a dedicated MySQL server machine, having excess memory will not be a problem, but when you are using shared server machine, having free memory will be useful for other applications and operating system.

You can use this command to check the memory status:

SHOW ENGINE INNODB STATUS;

The sample output will be like as follows:


BUFFER POOL AND MEMORY

Total large memory allocated 26386366464
Dictionary memory allocated 23826297
Buffer pool size 1572672
Free buffers 8192
Database pages 1553364
Old database pages 573246
Modified db pages 36
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 881819, not young 18198964
0.02 youngs/s, 0.05 non-youngs/s
Pages read 681064, created 2749237, written 3988300
0.02 reads/s, 0.12 creates/s, 11.50 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 1553364, unzip_LRU len: 0
I/O sum[5152]:cur[0], unzip sum[0]:cur[0]

Free buffers:

Having higher value for Free buffers is an indicator of you are having free InnoDB buffer pool. But make sure that, you are checking this value many times before coming to the conclusion. If you are seeing this higher value for a longer period of time, you can consider to decreasing the InnoDB buffer pool size.

InnoDB buffer pool hit ratio:

Buffer pool hit ratio = innodb_buffer_pool_read_requests / (innodb_buffer_pool_read_requests + innodb_buffer_pool_reads ) * 100

Innodb_buffer_pool_hit_ration below – 99.9% indicates that – innodb_buffer_pool_could be increased.

  • What are the InnoDB buffer pool status variables?

You can view the InnoDB buffer pool status variables by running this command:

show global status like ‘%innodb_buffer_pool_pages%’;

The sample output will be:

mysql> show global status like ‘%innodb_buffer_pool_pages%’;
+———————————-+———+
| Variable_name | Value |
+———————————-+———+
| Innodb_buffer_pool_pages_data | 1553363 |
| Innodb_buffer_pool_pages_dirty | 14 |
| Innodb_buffer_pool_pages_flushed | 3994981 |
| Innodb_buffer_pool_pages_free | 8193 |
| Innodb_buffer_pool_pages_misc | 11116 |
| Innodb_buffer_pool_pages_total | 1572672 |
+———————————-+———+
6 rows in set (0.03 sec)

Explanation:

innodb_buffer_pool_pages_data shows the number of dirty and clean data and index pages.

innodb_buffer_pool_pages_misc shows the number of pages that are busy because they have been allocated for administrative overhead such as row locks or the adaptive hash index.

innodb_buffer_pool_pages_free shows the free pages in InnoDB buffer pool – a large number of free pages over a longer period is a strong indicator that InnoDB buffer pool is too big and can easily be decreased.

innodb_buffer_pool_pages_dirty shows the number of InnoDB buffer pool data pages that are modified in memory but not yet written to the data files (dirty page flushing).

innodb_buffer_pool_pages_flushed: indicates the number of requests to flush the dirty pages from the InnoDB buffer pool.

innodb_buffer_pool_reads: it indicates the number of requests that cannot be satisfied with InnoDB buffer pool. Need to read from the disk.

innodb_buffer_pool_read_requests: it indicates the number of requests of logical reads from memory.

innodb_buffer_pool_wait_free: it indicates the number of times, queries has to wait for dirty pages to be flushed to disk. It is a counter, it counts how many times this flush has happened. If innodb_buffer_pool_wait_free > 0 is a strong indicator that the InnoDB buffer pool is too small.

innodb_buffer_pool_write_request – indicates the number of writes done to the buffer pool.The ratio of write requests to pages flushed indicates the number of rows changed in a block before it flushed to disk

Rows changes per flush = innodb_buffer_pool_write_request / innodb_buffer_pool_pages_flushed
Sample output:
mysql> show global status like ‘%innodb_buffer_pool_pages_data%’;
+——————————-+———+
| Variable_name | Value |
+——————————-+———+
| Innodb_buffer_pool_pages_data | 1527965 |
+——————————-+———+
1 row in set (0.59 sec)

mysql> show global status like ‘%innodb_buffer_pool_pages_misc%’;
+——————————-+——-+
| Variable_name | Value |
+——————————-+——-+
| Innodb_buffer_pool_pages_misc | 44863 |
+——————————-+——-+
1 row in set (0.55 sec)

mysql> show global status like ‘%innodb_buffer_pool_pages_free%’;
+——————————-+——-+
| Variable_name | Value |
+——————————-+——-+
| Innodb_buffer_pool_pages_free | 92 |
+——————————-+——-+
1 row in set (0.58 sec)

mysql> show global status like ‘%innodb_buffer_pool_pages_dirty%’;
+——————————–+——-+
| Variable_name | Value |
+——————————–+——-+
| Innodb_buffer_pool_pages_dirty | 757 |
+——————————–+——-+
1 row in set (0.49 sec)

mysql> show global status like ‘%innodb_buffer_pool_pages_flushed%’;
+———————————-+——–+
| Variable_name | Value |
+———————————-+——–+
| Innodb_buffer_pool_pages_flushed | 839273 |
+———————————-+——–+
1 row in set (0.59 sec)

mysql> show global status like ‘%innodb_buffer_pool_reads%’;
+————————–+———+
| Variable_name | Value |
+————————–+———+
| Innodb_buffer_pool_reads | 1684867 |
+————————–+———+
1 row in set (0.58 sec)

mysql> show global status like ‘%innodb_buffer_pool_read_requests%’;
+———————————-+————-+
| Variable_name | Value |
+———————————-+————-+
| Innodb_buffer_pool_read_requests | 26439817181 |
+———————————-+————-+
1 row in set (0.61 sec)

mysql> show global status like ‘%innodb_buffer_pool_wait_free%’;
+——————————+——-+
| Variable_name | Value |
+——————————+——-+
| Innodb_buffer_pool_wait_free | 0 |
+——————————+——-+
1 row in set (0.51 sec)

mysql> show global status like ‘%innodb_buffer_pool_write_request%’;
+———————————–+———-+
| Variable_name | Value |
+———————————–+———-+
| Innodb_buffer_pool_write_requests | 49063623 |
+———————————–+———-+
1 row in set (0.52 sec)

Some useful queries:
How much memory should allocate to InnoDB buffer pool?

Recommended InnoDB buffer pool size based on the all InnoDB data and indices with additional 50% memory:

set @idbdataindx = (select sum(data_length+index_length) from information_schema.tables where engine = ‘innodb’);

set @ibpsG = @idbdataindx * 1.5 / (102410241024);

select @ibpsG;

Sample output:

mysql> set @idbdataindx = (select sum(data_length+index_length) from information_schema.tables where engine = ‘innodb’);
Query OK, 0 rows affected, 1 warning (42.17 sec)

mysql> select @idbdataindx;
+————–+
| @idbdataindx |
+————–+
| 251380645888 |
+————–+
1 row in set (0.50 sec)

mysql> set @ibpsG = @idbdataindx * 1.5 / (102410241024);
Query OK, 0 rows affected (0.50 sec)

mysql> select @ibpsG;
+—————+
| @ibpsG |
+—————+
| 351.174705505 |
+—————+
1 row in set (0.61 sec)

  • How much actual GB of memory is in use by InnoDB buffer pool at this moment?

By multiplying data available in the buffer pool and InnoDB page (InnoDB buffer pool unit) size we can find the actual memory is in use by InnoDB buffer pool at this moment.

set @ibpdata = (select variable_value from information_schema.global_status where variable_name = ‘innodb_buffer_pool_pages_data’);

set @idbpgsize = (select variable_value from information_schema.global_status where variable_name = ‘innodb_page_size’);

set @ibpsize = @ibpdata * @idbpgsize / (102410241024);

select @ibpsize;

Sample output:

mysql> set @ibpdata = (select variable_value from information_schema.global_status where variable_name = ‘innodb_buffer_pool_pages_data’);

Query OK, 0 rows affected (0.50 sec)

mysql> set @idbpgsize = (select variable_value from information_schema.global_status where variable_name = ‘innodb_page_size’);

Query OK, 0 rows affected (0.51 sec)

mysql> set @ibpsize = @ibpdata * @idbpgsize / (102410241024);

Query OK, 0 rows affected (0.51 sec)

mysql> select @ibpsize;
+——————–+
| @ibpsize |
+——————–+
| 23.310394287109375 |
+——————–+
1 row in set (0.50 sec)

Note: In MySQL 5.7 the global status table moved to the performance_schema database. By default, it is disabled on the information_schema database.

Finally, there is no single solution for everyone, you have to understand your environment and decide wisely. Searching on the internet will give you some insights but it may not directly suitable to your environment. One thing I can say is, you have start somewhere, so learn it and master the task. I hope this post will give you some idea about InnoDB buffer pool. If you feel, I missed something here / any doubt or clarification on this post, please mention on the comment section.

Have you ever use @Autowired HttpServletRequest in Spring? At first glance, It seems quite strange, since we know basically there are 4 kinds of variables in Spring: Singleton, Protototype, Request, Session. I thought it’s a singleton instance, so it must be wrong to use the injection in such way. But, after some googling, It turns out that it’s OK to use in such a  way. But the tutorial or the answers from stackoverflow just says that the @autowired HttpServeltRequest will load the actual current request whenever we actually use the variable. But they apparently do not mention why, so I come with my own exploring of spring source code to find the reason. The version of spring I choose is 4.3.7.RELEASE. The conjunction of the spring-based container with the servlet-context container is ContextLoaderListener . Whenever the web context is bootstrapped, it will initialize the servlet context. We often use ServletContextListenerto register our self-defined business logic for the servlet context. So does the spring application context. Inside the ContextLoaderListener bootstrap and shutdown spring WebApplicationContext. As you can see down below:
  1. initialize the context
/**
 * Initialize the root web application context.
 */
@Override
public void contextInitialized(ServletContextEvent event) {
  initWebApplicationContext(event.getServletContext());
}


/**
 * Close the root web application context.
 */
@Override
public void contextDestroyed(ServletContextEvent event) {
  closeWebApplicationContext(event.getServletContext());
  ContextCleanupListener.cleanupAttributes(event.getServletContext());
}
 
  1. configure the webapplicationcontext
public WebApplicationContext initWebApplicationContext(ServletContext servletContext) {
        // ...
        
        try {
            // Store context in local instance variable, to guarantee that
            // it is available on ServletContext shutdown.
            if (this.context == null) {
                this.context = createWebApplicationContext(servletContext);
            }
            if (this.context instanceof ConfigurableWebApplicationContext) {
                ConfigurableWebApplicationContext cwac = (ConfigurableWebApplicationContext) this.context;
                if (!cwac.isActive()) {
                    // The context has not yet been refreshed -> provide services such as
                    // setting the parent context, setting the application context id, etc
                    if (cwac.getParent() == null) {
                        // The context instance was injected without an explicit parent ->
                        // determine parent for root web application context, if any.
                        ApplicationContext parent = loadParentContext(servletContext);
                        cwac.setParent(parent);
                    }
                    configureAndRefreshWebApplicationContext(cwac, servletContext);
                }
            }  
       }

}


  1. refresh the context
protected void configureAndRefreshWebApplicationContext(ConfigurableWebApplicationContext wac, ServletContext sc) {
    // any initialization logic
    customizeContext(sc, wac);
    wac.refresh();
}
  And that’s the whole process of the initialization logic of spring based web application.   So Let’s take a little deeper. What does the  wac.refresh()method do? I find the implementation in  AbstractApplicationContext.java file. Here is the actual logic:
      @Override
public void refresh() throws BeansException, IllegalStateException {
  synchronized (this.startupShutdownMonitor) {
    // Prepare this context for refreshing.
    prepareRefresh();

    // Tell the subclass to refresh the internal bean factory.
    ConfigurableListableBeanFactory beanFactory = obtainFreshBeanFactory();

    // Prepare the bean factory for use in this context.
    prepareBeanFactory(beanFactory);

    // Allows post-processing of the bean factory in context subclasses.
    postProcessBeanFactory(beanFactory);

    // Invoke factory processors registered as beans in the context.
    invokeBeanFactoryPostProcessors(beanFactory);

    // Register bean processors that intercept bean creation.
    registerBeanPostProcessors(beanFactory);

    // Initialize message source for this context.
    initMessageSource();

    // Initialize event multicaster for this context.
    initApplicationEventMulticaster();

    // Initialize other special beans in specific context subclasses.
    onRefresh();

    // Check for listener beans and register them.
    registerListeners();
    // Instantiate all remaining (non-lazy-init) singletons.
    finishBeanFactoryInitialization(beanFactory);

    // Last step: publish corresponding event.
    finishRefresh();
  }
}
  Basically, it is about 9 steps that we should do to start up a spring container.   We choose the implementation of the postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory)method. The web related context of spring is AbstractRefreshableWebApplicationContext. we find the implementation of  postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory).
@Override
  protected void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) {
    beanFactory.addBeanPostProcessor(new ServletContextAwareProcessor(this.servletContext, this.servletConfig));
    beanFactory.ignoreDependencyInterface(ServletContextAware.class);
    beanFactory.ignoreDependencyInterface(ServletConfigAware.class);

    WebApplicationContextUtils.registerWebApplicationScopes(beanFactory, this.servletContext);
    WebApplicationContextUtils.registerEnvironmentBeans(beanFactory, this.servletContext, this.servletConfig);
  }
  Let’s take a little deeper, go to the registerWebApplicationScopes.
public static void registerWebApplicationScopes(ConfigurableListableBeanFactory beanFactory, ServletContext sc) {
    beanFactory.registerScope(WebApplicationContext.SCOPE_REQUEST, new RequestScope());
    beanFactory.registerScope(WebApplicationContext.SCOPE_SESSION, new SessionScope(false));
    beanFactory.registerScope(WebApplicationContext.SCOPE_GLOBAL_SESSION, new SessionScope(true));
    if (sc != null) {
      ServletContextScope appScope = new ServletContextScope(sc);
      beanFactory.registerScope(WebApplicationContext.SCOPE_APPLICATION, appScope);
      // Register as ServletContext attribute, for ContextCleanupListener to detect it.
      sc.setAttribute(ServletContextScope.class.getName(), appScope);
    }

    beanFactory.registerResolvableDependency(ServletRequest.class, new RequestObjectFactory());
    beanFactory.registerResolvableDependency(ServletResponse.class, new ResponseObjectFactory());
    beanFactory.registerResolvableDependency(HttpSession.class, new SessionObjectFactory());
    beanFactory.registerResolvableDependency(WebRequest.class, new WebRequestObjectFactory());
    if (jsfPresent) {
      FacesDependencyRegistrar.registerFacesDependencies(beanFactory);
    }
  }
See, at the bootstrap phase, the spring context will register 4 kinds of different kinds of variables for their own resolve logic. The above code just says that if you want to find a bean of type ServletRequest, just look for the RequestObjectFactory instance. It will handle all the real retrieve. Let’s dig into RequestObjectFactory:  
private static class RequestObjectFactory implements ObjectFactory, Serializable {

    @Override
    public ServletRequest getObject() {
      return currentRequestAttributes().getRequest();
    }

    @Override
    public String toString() {
      return "Current HttpServletRequest";
    }
  }


private static ServletRequestAttributes currentRequestAttributes() {
   RequestAttributes requestAttr = RequestContextHolder.currentRequestAttributes();
   if (!(requestAttr instanceof ServletRequestAttributes)) {
      throw new IllegalStateException("Current request is not a servlet request");
   }
   return (ServletRequestAttributes) requestAttr;
}

From above, we know that every time we want to read the value of ServletRequest bean, the spring container will delegate it to RequestObjectFactory, and delegate to RequestContextHolder. Let’s dig into RequestContextHolder:  
public static RequestAttributes currentRequestAttributes() throws IllegalStateException {
    RequestAttributes attributes = getRequestAttributes();
    // ... jsf special feature
    return attributes;
  }

public static RequestAttributes getRequestAttributes() {
    RequestAttributes attributes = requestAttributesHolder.get();
    if (attributes == null) {
      attributes = inheritableRequestAttributesHolder.get();
    }
    return attributes;
  }
private static final ThreadLocal requestAttributesHolder =
      new NamedThreadLocal("Request attributes");
Follow the chain, we know that the currentRequestAttributes() is actually requesting a static ThreadLocal variable. Till now, we know that every time when we try to access the HttpServletRequest variable, we are just requesting a value that’s stored inside of a static ThreadLocal variable.     But, since when do we store our request inside a static ThreadLocal variable?   Let’s follow the chain from setting the requestAttributesHolder:
public static void setRequestAttributes(RequestAttributes attributes) {
    setRequestAttributes(attributes, false);
  }
By use of code assistance, we know that there are FrameworkServlet, RequestContextListener that try to use the above method to set up the ThreadLocal variable besides the testing class.   Let’s see the RequestContextListenerfirst. The RequestContextListenerimplements ServletRequestListener. ServletRequestListener is a servlet standard interface that fulfills the developer’s need to modify the request in and out of a web component. Here we can expose the request to the current thread. As the doc says, the RequestContextListener is maily for the use with third-party servelts, e.g. the JSF FacesServlet. Within Spring’s own web support, DispatcherServelt’s processing is perfercely sufficient.   So, let’s see the FrameworkServlet. FrameworkServlet is the base servlet for Spring’s web framework. It’s just like an ordinary java servlet. So let’s take doGet entrance for example.
@Override
  protected final void doGet(HttpServletRequest request, HttpServletResponse response)
      throws ServletException, IOException {

    processRequest(request, response);
  }


protected final void processRequest(HttpServletRequest request, HttpServletResponse response)
      throws ServletException, IOException {

   long startTime = System.currentTimeMillis();
   Throwable failureCause = null;

   LocaleContext previousLocaleContext = LocaleContextHolder.getLocaleContext();
   LocaleContext localeContext = buildLocaleContext(request);

   RequestAttributes previousAttributes = RequestContextHolder.getRequestAttributes();
   ServletRequestAttributes requestAttributes = buildRequestAttributes(request, response, previousAttributes);

   WebAsyncManager asyncManager = WebAsyncUtils.getAsyncManager(request);
   asyncManager.registerCallableInterceptor(FrameworkServlet.class.getName(), new RequestBindingInterceptor());

   initContextHolders(request, localeContext, requestAttributes);

   
   doService(request, response);
   
}

protected ServletRequestAttributes buildRequestAttributes(
      HttpServletRequest request, HttpServletResponse response, RequestAttributes previousAttributes) {

   if (previousAttributes == null || previousAttributes instanceof ServletRequestAttributes) {
      return new ServletRequestAttributes(request, response);
   }
   else {
      return null;  // preserve the pre-bound RequestAttributes instance
   }
}
  As you can see from processRequest method, every time the raw HttpServletRequest comes, it will first try to retrieve the previous request, if the request is null, create one, then call the initContextHolders method. Let’s dig into initContextHolders method:
private void initContextHolders(
      HttpServletRequest request, LocaleContext localeContext, RequestAttributes requestAttributes) {

    if (localeContext != null) {
      LocaleContextHolder.setLocaleContext(localeContext, this.threadContextInheritable);
    }
    if (requestAttributes != null) {
      RequestContextHolder.setRequestAttributes(requestAttributes, this.threadContextInheritable);
    }
    if (logger.isTraceEnabled()) {
      logger.trace("Bound request context to thread: " + request);
    }
  }
  Finally, we solve the question we mention earlier. we discover that every time when a raw http request comes, the Spring will try to set the current http servlet request into a global static ThreadLocal variable so that later the request can be retrieve back and forth even in mutli-thread evvironment.        

basic command

  • Initialize the cluster
    sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.33.242 --kubernetes-version stable-1.11 -v 10
    
  • View the logs
    kubectl logs -f kubernetes-dashboard-767dc7d4d-f8r8l -n kube-system
    
  • Forward the port
    kubectl port-forward pod/guids-796989bbbb-qj2nl 9000:9000
    
  • Expose the port (By NodePort)
    kubectl expose deployment/kubernetes-bootcamp --type="NodePort" --port 8080
    
    kubectl expose deployments hello-world
    
  • Label a pod
    kubectl label pod guids-796989bbbb-qj2nl app=guids
    
    kubectl get pods -l app=guids
    
  • Tear down the cluster
    kubectl drain neo --delete-local-data --force --ignore-daemonsets
    kubectl delete pod,service baz foo
    
  • Re-add the node
    service kubelet restart
    kubectl uncordon neo
    
  • Various get
    kubectl get nodes
    kubectl get pods
    kubectl get services
    kubectl get deployments -o wide
    kubectl get all -n kube-system
    kubectl get storageclass
    kubectl api-resources
    
  • Describe
    kubectl describe services guids
    
  • Create a deployment
    kubectl run hello-world --replicas=2 --labels="run=load-balancer-example" --image=neocxf/hello-world  --port=8080
    
  • View the pod name
    export POD_NAME=$(kubectl get pods -o go-template --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
    
  • Configure default storage class
    kubectl patch storageclass standard -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
    
    kubectl create -f https://k8s.io/examples/pods/storage/pv-volume.yaml
    kubectl get pv task-pv-volume
    kubectl create -f https://k8s.io/examples/pods/storage/pv-claim.yaml
    kubectl get pvc task-pv-claim
    
  • How to set a pv and pvc
    kubectl scale sts web --replicas=2
    kubectl delete pvc www-web-2
    kubectl patch pv task-pv-volume3 # delete pvc in pv yaml
    kubectl get pv                   # now we have an available pv
    
  • Print the join command
    kubeadm token create --print-join-command
    
  • ConfigMap
    kubectl create configmap example-redis-config --from-file=redis-config
    kubectl get configmap example-redis-config -o yaml
    
  • Redis-Master-Slave-Frontend deploy

2. How Kubernetes work

2.1 Main components

kubernetes-flow