https://jerrywang1981.github.io/tags/it/ Recent content in It on 村边的池塘 Hugo zh-CN Mon, 10 Apr 2023 14:34:26 +0800 https://jerrywang1981.github.io/post/vim/fzf-vim/ Mon, 10 Apr 2023 14:34:26 +0800 https://jerrywang1981.github.io/post/vim/fzf-vim/ configure ctrl + p to search files if in git repo, then call :GFiles, otherwise call :Files function <SID>CtrlP() let l:j1 = system("git -C " . getcwd() . " rev-parse --is-inside-work-tree") let l:j2 = system("git -C " . getcwd() . " rev-parse --is-bare-repository") if l:j1 !~ "true" && l:j2 !~ "true" :Files else :GFiles endif endfunction nnoremap <silent> <C-p> :call <SID>CtrlP()<CR> nnoremap <silent> <leader>fb :Buffers<CR> nnoremap <silent> <leader>fh :History<CR> nnoremap <silent> <leader>fs :Rg<CR> nnoremap <silent> <leader>fS :Rg <c-r><c-w><CR> https://jerrywang1981.github.io/post/k8s/413-Request-Entity-Too-Large/ Tue, 04 Apr 2023 16:34:00 +0800 https://jerrywang1981.github.io/post/k8s/413-Request-Entity-Too-Large/ Error message 413 request entity too large How to fix Nginx increase client_max_body_size Kubenetes Ingress increase proxy-body-size reference -> https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/ https://jerrywang1981.github.io/post/node/axios-error-Request-body-larger-than-maxBodyLength-limit/ Tue, 28 Mar 2023 18:52:12 +0800 https://jerrywang1981.github.io/post/node/axios-error-Request-body-larger-than-maxBodyLength-limit/ Axios error Request Body Larger Than MaxBodyLength Limit solution config axios maxContentLength/maxBodyLength to a large number or Infinity const options = { url, //method: 'post', //... maxContentLength: Infinity, maxBodyLength: Infinity, }; https://jerrywang1981.github.io/post/other/address-difficult-situation/ Mon, 27 Feb 2023 11:16:14 +0800 https://jerrywang1981.github.io/post/other/address-difficult-situation/ 概述 工作中经常会遇到各种各样的困难，或者是艰难场景，如果处理不好，会对同事，工作，项目造成影响。 常见的一些类型 目标冲突 一个人给出的目标与另一个人共享的目标相冲突, 比如员工的个人职业规划和安排的工作目标有冲突。 角色冲突 当某人要执行一项不属于他们通常的角色或职责的任务时，或者当他们被 https://jerrywang1981.github.io/post/security/Cacheable-SSL-Page-Found/ Mon, 27 Feb 2023 09:27:53 +0800 https://jerrywang1981.github.io/post/security/Cacheable-SSL-Page-Found/ Issue Cacheable SSL Page Found Risk It is possible to gather sensitive information about the web application such as usernames, passwords, machine name and/or sensitive file locations Cause Sensitive information might have been cached by your browser Fix Prevent caching of SSL pages by adding “Cache-Control: no-store” and “Pragma: no-cache” headers to their responses. Possible solution you can also change the HTTP method from GET to POST if it is NOT a page but a rest API. https://jerrywang1981.github.io/post/security/Cookie-with-Insecure-or-Improper-or-Missing-SameSite-attribute/ Mon, 27 Feb 2023 09:15:23 +0800 https://jerrywang1981.github.io/post/security/Cookie-with-Insecure-or-Improper-or-Missing-SameSite-attribute/ Issue Cookie with Insecure or Improper or Missing SameSite attribute Risk Prevent cookie information leakage by restricting cookies to first-party or same-site context, Attacks can extend to Cross-Site- Request-Forgery (CSRF) attacks if there are no additional protections in place (such as Anti-CSRF tokens). Cause Sensitive Cookie with Improper or Insecure or Missing SameSite Attribute Fix Review possible solutions for configuring SameSite Cookie attribute to recommended values for K8s, you may edit the ingress configuration, set samesite value nginx.ingress.kubernetes.io/session-cookie-samesite: "Strict" https://jerrywang1981.github.io/post/node/blind-sql-injection/ Wed, 16 Nov 2022 09:29:06 +0800 https://jerrywang1981.github.io/post/node/blind-sql-injection/ Security Risk: It is possible to view, modify or delete database entries and tables Cause: Sanitation of hazardous characters was not performed correctly on user input Fix: Review possible solutions for hazardous character injection Reasoning: The test result seems to indicate a vulnerability because it shows that values can be appended to parameter values, indicating that they were embedded in an SQL query. In this test, three (or sometimes four) requests are sent. The last is logically equal to the original, and the next-to-last is different. Any others are for control purposes. A comparison of the last two responses with the first (the last is similar to it, and the next-to-last is different) indicates that the application is vulnerable. How to fix https://sequelize.org/v5/manual/raw-queries.html make sure to ‘replace’/‘bind’ https://jerrywang1981.github.io/post/go/golang-oauth2-github/ Fri, 09 Apr 2021 20:40:45 +0800 https://jerrywang1981.github.io/post/go/golang-oauth2-github/ Github Oauth2 首先，在github上申请一个Oauth App, 获取client id / secret 需要这些信息，下边的代码是放在环境变量中 AUTH_URL = os.Getenv("AUTH_URL") // github authorization url TOKEN_URL = os.Getenv("TOKEN_URL") // github token url CLIENT_ID = os.Getenv("CLIENT_ID") // client id CLIENT_SECRET = os.Getenv("CLIENT_SECRET") // client secret REDIRECT_URL = os.Getenv("REDIRECT_URL") // the redirect url 在包的初始化代码中，初始化 var ( oauth2Config *oauth2.Config ) init() { oauth2Config = &oauth2.Config{ ClientID: CLIENT_ID, ClientSecret: CLIENT_SECRET, RedirectURL: REDIRECT_URL, // Discovery returns the OAuth2 endpoints. Endpoint: oauth2.Endpoint{ AuthURL: AUTH_URL, TokenURL: TOKEN_URL, AuthStyle: oauth2.AuthStyleInHeader, }, Scopes: []string{"openid", "email", "site_admin", "repo", "admin:org", "user"}, } Login https://jerrywang1981.github.io/post/go/terminate-golang-app/ Sat, 06 Feb 2021 16:10:08 +0800 https://jerrywang1981.github.io/post/go/terminate-golang-app/ channel to terminate 在 main.go中, 创建一个channel接收os.Signal sigChan := make(chan os.Signal, 1) // ctrl+c->SIGINT, kill -9 -> SIGKILL signal.Notify(sigChan, syscall.SIGINT, syscall.SIGKILL) 在main.go快结束的位置，加上 <-sigChan log.Info("exit") 如果用户有发送kill -9 或者ctrl+c， 会接收到这个信号。 https://jerrywang1981.github.io/post/go/golang-cron-jobs/ Sat, 06 Feb 2021 16:06:53 +0800 https://jerrywang1981.github.io/post/go/golang-cron-jobs/ cron in golang 在golang中，有个包github.com/robfig/cron/v3可以实现linux中类似的cron功能，使用也比较简单。 官方文档中比较详细，这里只是简单记录一下。 c := cron.New() schlTime := "0 2 * * ?" // send reminder for org members c.AddFunc(schlTime, func() { log.Info("Send reminder for org member expiration job started") defer func() { log.Info("Send reminder for org member expiration job finished") }() log.Info("started to remind expired org members") err = task.RemindExpiredOrgMember(14) if err != https://jerrywang1981.github.io/post/go/gin-jwt-middleware/ Sat, 06 Feb 2021 14:56:43 +0800 https://jerrywang1981.github.io/post/go/gin-jwt-middleware/ JWT in golang golang中的jwt包一般是使用 github.com/dgrijalva/jwt-go, 而且godoc中已经有些example，这里只是记录一下使用 jwt.StandardClaims的情况。 func parseToken(tokenString string) (*jwt.StandardClaims, error) { token, err := jwt.ParseWithClaims(tokenString, &jwt.StandardClaims{}, func(t *jwt.Token) (interface{}, error) { return jwtKey, nil }) if err != nil { log.Errorf("error: %v", err) return nil, fmt.Errorf("failed to parse the token string: %s", tokenString) } log.Printf("claims: %v", token.Claims) if claims, ok := token.Claims.(*jwt.StandardClaims); ok && token.Valid { return claims, nil } else { return nil, errors.Wrap(fmt.Errorf("failed to valiate the token"), "failed") } } https://jerrywang1981.github.io/post/java/liberty-profile-tls-v1-gaps/ Thu, 12 Nov 2020 19:31:52 +0800 https://jerrywang1981.github.io/post/java/liberty-profile-tls-v1-gaps/ Gap - TLS version 1.0/1.1 detected 如果在做security scan的时候，有这两个gap,需要修改两个地方 liberty profile server.xml add sslProtocol="TLSv1.2" to ssl <ssl id="defaultSSLConfig" sslProtocol="TLSv1.2" keyStoreRef="defaultKeyStore" securityLevel="CUSTOM" enabledCiphers="TLS_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA256 "/> server.env add JVM_ARGS=-Dhttps.protocols=TLSv1.2 https://jerrywang1981.github.io/post/react/react-for-beginners/ Fri, 25 Sep 2020 07:56:47 +0800 https://jerrywang1981.github.io/post/react/react-for-beginners/ Prerequesite node javascript html css es6 code editor create-react-app npm install -g create-react-app create-react-app app-name React component state class component里面管理状态使用state class Counter extends Component { state = { counter: 10, title: "this is demo", products: [ { id: 1, name: "ok", }, { id: 2, name: "bad", }, ], }; // ..... other } render jsx expression attributes list/array conditional rendering handle events binding event handlers updating state event arguments passing data to component passing children debug react app props vs state raising and handle events multiple component in sync lifting state up functional component destructure arguments lifecycle hooks https://jerrywang1981.github.io/post/java/add-third-party-jar-to-spring-boot/ Fri, 11 Sep 2020 15:55:30 +0800 https://jerrywang1981.github.io/post/java/add-third-party-jar-to-spring-boot/ 背景 有时候需要添加第三方的jar包到spring boot工程中，因为不能从maven repository直接下载，所以需要包含在代码库中。 一般需要在 src/main 目录下，可以建个新目录 lib, 然后把jar文件放到这个目录下，也就是 src/main/lib下. 然后在pom.xml中需要添加 group https://jerrywang1981.github.io/post/db/postgres-backup-and-restore/ Fri, 11 Sep 2020 15:14:50 +0800 https://jerrywang1981.github.io/post/db/postgres-backup-and-restore/ 背景 有时候我们需要备份 schema或者包括数据，然后恢复到另外一个postgres db中。 备份 可以使用 pg_dump 来备份，或者加上 –schema-only 只备份schema. pg_dump --file "/root/repo/uat-copy.tar" --host "db host name" --port "23123" --username "admin" --password --verbose --format=t --blobs --schema-only "compose" 恢复 同样的，可以使用pg_restore来恢复 pg_restore --host "hostname here" --port "31090" --username "usernamehere" --password --dbname "ibmclouddb" --verbose "/root/repo/uat-copy.tar" 不管备份还是恢复，都会要求输入密 https://jerrywang1981.github.io/post/linux/auto-login-server-without-password/ Fri, 11 Sep 2020 10:38:33 +0800 https://jerrywang1981.github.io/post/linux/auto-login-server-without-password/ 配置方法，做个小记录 server 上要配置允许 用户电脑 在 .ssh/config 文件中 Host * AddKeysToAgent yes UseKeychain yes IdentityFile ~/.ssh/id_rsa Host xxx HostName xx.xx.xx.xx User jerry 把pub key传到server ssh-copy-id -i ~/.ssh/id_rsa.pub xx@xx.xx.xx.x 可以了 https://jerrywang1981.github.io/post/vue/file-download-from-spring-boot/ Thu, 10 Sep 2020 07:52:49 +0800 https://jerrywang1981.github.io/post/vue/file-download-from-spring-boot/ File download 如果spring boot的写法是我在另外一个文章, 返回的response其实是一个Blob, 这一点要记住 this.axios.get(url, { responseType: 'blob', timeout: 0 }).then(response => { // response is Blob <------- const url = window.URL.createObjectURL(response) const a = document.createElement('a') a.style.display = 'none' a.href = url a.setAttribute('download','file.xlsx') document.body.appendChild(a) a.click() document.body.removeChild(a); window.URL.revokeObjectURL(url); }); https://jerrywang1981.github.io/post/java/spring-boot-file-download/ Thu, 10 Sep 2020 07:52:13 +0800 https://jerrywang1981.github.io/post/java/spring-boot-file-download/ spring boot application 提供下载的代码，可以返回一个Resource @RestController @RequiredArgsConstructor @RequestMapping("/api") public class ReportController { @GetMapping(value = "/filedownload") public ResponseEntity<InputStreamResource> downloadFile(HttpServletRequest request) { // ....................... // ....................... // ....................... ByteArrayInputStream stream = xxxxxx;// get the stream return ResponseEntity.ok().contentType(MediaType.APPLICATION_OCTET_STREAM) .header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=file.xlsx") .body(new InputStreamResource(stream)); } } https://jerrywang1981.github.io/post/other/azure-saml-sso-node-app/ Thu, 27 Aug 2020 22:05:59 +0800 https://jerrywang1981.github.io/post/other/azure-saml-sso-node-app/ Azure AD saml sso 代码 主要是参考文档 安装 passport-saml npm install passport-saml use passport passport.use(new SamlStrategy( { path: '/account/auth/blueid/callback', // 这个是callback需要稍后实现 entryPoint: '', // 这个是azure AD的一个配置, saml-based sso login url issuer: 'cobeedev', // basic saml configuration -> Identifier (Entity ID) cert: '', // refer later signatureAlgorithm: 'sha256' }, async function(profile, done) { // in this function, you can process profile to get necessary information console.log(profile); // lazy require const usersManagement = require('../../controllers/users/usersManagement'); const ssoUser = jsonUtil.profile2User(profile); let dbUser = await usersManagement.getUserByAzureMail(ssoUser.email); // if user was not found in db // process the scenario if (!dbUser) { await usersManagement.processAzureSSOUser(ssoUser); dbUser = await usersManagement.getUserByAzureMail(ssoUser.email); } https://jerrywang1981.github.io/post/other/port-forwarding/ Thu, 27 Aug 2020 21:27:22 +0800 https://jerrywang1981.github.io/post/other/port-forwarding/ Mac下端口转发 有时候要在本地起https,需要把443端口转发到8080或者其他的3000端口 ###设置转发 echo "rdr pass inet proto tcp from any to any port 443 -> 127.0.0.1 port 9443" | sudo pfctl -ef - 取消转发 sudo pfctl -s nat sudo pfctl -F all -f /etc/pf.conf https://jerrywang1981.github.io/post/go/zero-value-in-golang/ Tue, 18 Aug 2020 09:09:46 +0800 https://jerrywang1981.github.io/post/go/zero-value-in-golang/ zero value (零值) 在golang中，零值会在很多地方隐式的用到，所以把零值列出来。 0 所有的integer类型 (int32/int64/…) 0.0 所有的float类型 (float32/float64/…) false boolean类型 "" string类型， 这点要特别注意，从java过来的开发人员容易认为 nil是把 string和nil比较， 应该 str == "" ,不是 str == nil nil interface, slices, https://jerrywang1981.github.io/post/go/golang-panic-recover-defer-update-return/ Mon, 17 Aug 2020 20:53:08 +0800 https://jerrywang1981.github.io/post/go/golang-panic-recover-defer-update-return/ 在panic的函数中更新返回值 在golang中我们知道可以用defer, panic, recover来处理一些情况，比如，一般panic的话，如果在函数中没有处理，就会向上继续抛出 有时候，我们希望在出错的情况下，能够返回一个值。这个时候，defer就会比较好的处理这种情况。 func test() (r int) { defer func() { if https://jerrywang1981.github.io/post/go/watson-assistant-easy-tool/ Sat, 15 Aug 2020 18:41:04 +0800 https://jerrywang1981.github.io/post/go/watson-assistant-easy-tool/ Watson assistant 根据官方文档，已经提供了sdk, 自己的这个package只是使用 官方的sdk基础上，做了小小的封闭。代码托管在github上 watson 目前只有一个assitant package,提供方法来开始一个session并且发送message从而得到结果。 使用方法也比较简单，NewSessio https://jerrywang1981.github.io/post/angular/multiple-angular-material-table-sort/ Tue, 04 Aug 2020 15:44:13 +0800 https://jerrywang1981.github.io/post/angular/multiple-angular-material-table-sort/ 问题 在一个component里面，如果有多个mat-table matSort,怎么使用ViewChild 方案 使用ViewChild的时候，需要使用id @ViewChild('sort1') sort1: MatSort; 在页面中，需要 <mat-table #table [dataSource]="dataSource" #sort1="matSort" matSort> https://jerrywang1981.github.io/post/angular/angular-material-table-paginator-ngif/ Tue, 04 Aug 2020 15:26:25 +0800 https://jerrywang1981.github.io/post/angular/angular-material-table-paginator-ngif/ 问题 一般来说，按angular material的官方文档, 在afterViewInit里面 可以把paginator或者sort和dataSource绑定在一起。但是如果这个table是在ngIf里面，也就是要在条件满足的情况下。这个时候的 @ViewChild(MatPaginator) paginator: MatPaginator; 可能就不好用了。因为还没有渲染，对应 https://jerrywang1981.github.io/post/go/go-download-from-private-repo-in-docker/ Fri, 24 Jul 2020 08:33:43 +0800 https://jerrywang1981.github.io/post/go/go-download-from-private-repo-in-docker/ 问题 有时候开发的go project会用到一些私有仓库，或者自己搭的。比如用到了github.ibm.com里面的代码，如果在go.mod里面设置的 是github.ibm.com的网址的话，会发现build docker image的时候，提示access rights的问题。 方法 在你的Docke https://jerrywang1981.github.io/post/go/go-gorm/ Thu, 23 Jul 2020 08:25:59 +0800 https://jerrywang1981.github.io/post/go/go-gorm/ Go/Gorm 操作 Postgres 虽然go提供了database/sql包来操作数据库，但是如果有个ORM的包，可以直接操作model来操作数据库，就像node里面的sequelize,是不是 也是挺好的。Gorm就实现了这个功能，这个包目前是github上star数最多的go orm包，包括中文资料，对小伙 https://jerrywang1981.github.io/post/angular/multiple-line-in-mat-tooltip/ Wed, 24 Jun 2020 08:26:03 +0800 https://jerrywang1981.github.io/post/angular/multiple-line-in-mat-tooltip/ angular material matTooltip 如果想多行显示，可以把tooltip的内容用空格分开，同时样式 ::ng-deep .mat-tooltip { white-space: pre-line !important; } https://jerrywang1981.github.io/post/python/18-matplotlib-demo/ Tue, 09 Jun 2020 09:08:07 +0800 https://jerrywang1981.github.io/post/python/18-matplotlib-demo/ the code in jupyter notebook matplot_demo https://jerrywang1981.github.io/post/python/17-pandas-demo-2/ Tue, 09 Jun 2020 09:05:34 +0800 https://jerrywang1981.github.io/post/python/17-pandas-demo-2/ the code in jupyter notebook pandas_demo.ipynb https://jerrywang1981.github.io/post/javascript/hugo-template-background-with-shapes/ Wed, 03 Jun 2020 21:30:06 +0800 https://jerrywang1981.github.io/post/javascript/hugo-template-background-with-shapes/ 就是现在网页的背景效果，是从网上抄的 需要把文件放到个js里面，然后在foolter里面引用。 代码如下, 本质是个IIFE !function () { function n(n, e, t) { return n.getAttribute(e) || t } function e(n) { return document.getElementsByTagName(n) } function t() { var t = e("script"), o = t.length, i = t[o - 1]; return { l: o, z: n(i, "zIndex", -1), o: n(i, "opacity", .5), c: n(i, "color", "0,0,0"), n: n(i, "count", 99) } } function o() { a = m.width = window.innerWidth || document.documentElement.clientWidth || document.body.clientWidth, c = m.height = window.innerHeight || document.documentElement.clientHeight || document.body.clientHeight } function i() { https://jerrywang1981.github.io/post/python/16-pandas-demo-1/ Wed, 03 Jun 2020 16:30:03 +0800 https://jerrywang1981.github.io/post/python/16-pandas-demo-1/ the code in jupyter notebook refer to file pandas_demo.ipynb https://jerrywang1981.github.io/post/python/15-numpy-demo/ Wed, 03 Jun 2020 16:28:51 +0800 https://jerrywang1981.github.io/post/python/15-numpy-demo/ the code in jupyter notebook the file numpy_demo.ipynb https://jerrywang1981.github.io/post/python/14-anaconda/ Wed, 03 Jun 2020 16:27:28 +0800 https://jerrywang1981.github.io/post/python/14-anaconda/ How to install new package conda install xxx list env conda env list activate one env conda activate xx deactivate env conda deactivate https://jerrywang1981.github.io/post/python/13-python-context-manager/ Sun, 31 May 2020 13:58:44 +0800 https://jerrywang1981.github.io/post/python/13-python-context-manager/ python context manager use contextlib/yield @contextmanager def send_mail_context(): '''context manager with send_mail_context() ''' try: # 1 doing work BEFORE the code goes into with block print('before yield') server = smtplib.SMTP(SMTP_SERVERS[0], 25) # 2 yield back to with/caller yield server except Exception as e: print(e) finally: # 4. doing cleanup work server.quit() print('in finally') use class __enter__, __exit__ # 2 implement contextmanager with class class SMTPCT(object): ''' 1. override __enter__ __exit__ 2. in __enter__, return value ''' def __init__(self): self.smtp_server = None def __enter__(self): # 1 doing work BEFORE the code goes into with block print('in enter') self.smtp_server = smtplib.SMTP(SMTP_SERVERS[0], 25) # 2 return back to with/caller return self.smtp_server def __exit__(self, type, value, traceback): # 4. doing cleanup work print('in exit') self.smtp_server.quit() https://jerrywang1981.github.io/post/python/12-python-smtp/ Sun, 31 May 2020 13:52:03 +0800 https://jerrywang1981.github.io/post/python/12-python-smtp/ how to send mail in python #!/usr/bin/env python3 # -*- coding: utf-8 -*- # smtplib -> https://docs.python.org/3/library/smtplib.html import smtplib # contextlib -> https://docs.python.org/3/library/contextlib.html from contextlib import contextmanager # email -> https://docs.python.org/3/library/email.examples.html#email-examples from email import encoders from email.header import Header from email.mime.text import MIMEText from email.utils import parseaddr, formataddr SMTP_SERVERS = ['your smtp servers'] def _format_addr(s): ''' format the address ''' name, addr = parseaddr(s) return formataddr((Header(name, 'utf-8').encode(), addr)) def generate_mail(from_addr, to_addr, cc_addr, subject, body, mime_type='plain'): '''plain/text text/html ''' msg = MIMEText(body, mime_type, 'utf-8') msg['From'] = _format_addr(from_addr) msg['To'] = ",".join([_format_addr(x) for x in to_addr]) msg['Cc'] = ",".join([_format_addr(x) for x in cc_addr]) msg['Subject'] = Header(subject, 'utf-8').encode() return msg def send_mail(from_addr, to_addr, cc_addr, subject, body, mime_type='plain'): server = smtplib.SMTP(SMTP_SERVERS[0], 25) server.set_debuglevel(1) msg = generate_mail(from_addr, to_addr, cc_addr, subject, body, mime_type) server.sendmail(from_addr, to_addr + cc_addr, msg.as_string()) server.quit() # 1. to implement context manager, use decorator @contextmanager # and yield @contextmanager def send_mail_context(): '''context manager with send_mail_context() ''' try: # 1 doing work BEFORE the code goes into with block print('before yield') server = smtplib. https://jerrywang1981.github.io/post/python/11-python-mro/ Sun, 31 May 2020 13:50:50 +0800 https://jerrywang1981.github.io/post/python/11-python-mro/ method resolution order #!/usr/bin/env python3 # -*- coding: utf-8 -*- # method resolution order # mro class A(object): def hello(self): print('class A says hello') class B(A): pass class C(A): # class variable BP_URL = 'https://xxxxx' def __init__(self): self.name = 'ok' @staticmethod def print_hello(name, age): ''' staticmethod cannot access any class or self variabel/method ''' pass @classmethod def say(cls): '''class method can access class level variable/method ''' print(cls.BP_URL) def hello(self): print('class C says hello') class D(C, B): pass if __name__ == '__main__': d = D() d.hello() # to print the mro of one class print(d.__class__.__mro__) https://jerrywang1981.github.io/post/python/10-dunder-methods/ Sun, 31 May 2020 13:49:16 +0800 https://jerrywang1981.github.io/post/python/10-dunder-methods/ dunder methods #!/usr/bin/env python3 # -*- coding: utf-8 -*- # double underscore variable/function # dunder class A(object): def __init__(self, number): self.number = number def __eq__(self, value): ''' a == b ''' return super().__eq__(value) def __ge__(self, value): ''' a >= b ''' return super().__ge__(value) def __gt__(self, value): ''' a > b ''' return super().__gt__(value) def __le__(self, value): ''' a < b ''' return super().__le__(value) def __ne__(self, value): ''' a != b ''' return super().__ne__(value) def __str__(self): '''print(a) ''' return super().__str__() def __repr__(self): return super().__repr__() def __len__(self): ''' len(a) ''' return self.number * 10 def __lt__(self, other): '''less than a < b ''' return self.number < other.number def __getattribute__(self, name): ''' print(a[the_key]) ''' return super().__getattribute__(name) def __setattr__(self, name, value): ''' a[the_key] = 5 ''' return super().__setattr__(name, value) def __delattr__(self, name): ''' delete a['the_key'] ''' return super().__delattr__(name) if __name__ == '__main__': ''' ''' # a = A(5) # b = A(6) # print(a < b) # print(len(a)) # a['b'] = 10 # c = a['c'] pass https://jerrywang1981.github.io/post/ml/darknet-too-many-or-too-few-labels/ Wed, 27 May 2020 10:27:04 +0800 https://jerrywang1981.github.io/post/ml/darknet-too-many-or-too-few-labels/ Too many or too few labels 如果在使用darknet过程中，遇到这个错误，很可能你的labels文件里面的label定义的有问题 label之间要不包含 label 名字不要出现在train/test的路径中 所以如果定义一个label是a, 很可能就不行，train里面包含a，所以还是定义的稍微长点。 https://jerrywang1981.github.io/post/python/09-python-decorators/ Wed, 27 May 2020 10:17:38 +0800 https://jerrywang1981.github.io/post/python/09-python-decorators/ 无参装饰器 #1. decorator without arguments def log_time_always(func): @wraps(func) def wrapped(*args, **kwargs): #1. get current time t1 = datetime.now() #2. run the function func(*args, **kwargs) #3. get current time t2 = datetime.now() print(f'in {func.__name__}, it took {t2-t1}') return wrapped 带参数的装饰器 #2. decorator with arguments, flag is True or False def log_time(flag): def wrapper(func): @wraps(func) def wrapped(*args, **kwargs): # if flag is true, we need to log the time if flag: #1. get current time t1 = datetime.now() #2. run the function func(*args, **kwargs) #3. get current time t2 = datetime.now() print(f'in {func.__name__}, it took {t2-t1}') else: func(*args, **kwargs) return wrapped return wrapper 本质 装饰器需要能够处理一个函数 无参 @decorator_1 def f(): pass 等同于 def f(): pass f = https://jerrywang1981.github.io/post/python/08-python-virtual-environment/ Tue, 19 May 2020 21:00:00 +0800 https://jerrywang1981.github.io/post/python/08-python-virtual-environment/ Details refer to offical document Virtual environment To create new virtual environment usually in folder venv python -m venv yourfoldername enter virtual environment source venv/bin/activate exit virtual environment deactivate Pip usage install package pip install packagename list installed packages pip list save installed packages to requirements.txt pip freeze > requirements.txt install packages from requirements.txt pip install -r requirements.txt https://jerrywang1981.github.io/post/node/sequelize-bulk-create-postgre/ Tue, 19 May 2020 20:00:37 +0800 https://jerrywang1981.github.io/post/node/sequelize-bulk-create-postgre/ Sequelize data model 在使用 bulkCreate的时候，如果想在返回的数据中包括生成的id, 需要加上option, { returning: true } return CertificationEndUsers.bulkCreate(usersToInsert, { returning: true }); 另外 postgresql中table name包含双引号和不包含的，竟然是不一样的两个table "users" users 比如这是两个table https://jerrywang1981.github.io/post/python/07-python-function/ Sat, 16 May 2020 09:17:08 +0800 https://jerrywang1981.github.io/post/python/07-python-function/ 函数 #!/usr/bin/env python3 # -*- coding: utf-8 -*- # def func, snake case # return value # no args, no return def add_two_number(): print('a') # 1 arg def add_two_number_1(x): return x + 5 # return value # return multiple value def two_number_2(x, y): c = x + y d = x - y return c, d # args def add_number(x, y, *args): print(f'x = {x}') print(f'y = {y}') print(f'args = {args} ') # kwargs # def add_number_1(x, y, *args, **kwargs): print(f'x = {x}') print(f'y = {y}') # f-string f'this is string, {x}' print(f'args = {args} ') print(f'kwargs = {kwargs} ') # wrong # def add_number_2(**kwargs, *args, x, y) # pass # default def add_number_3(x, y, c=10, d = True): print(c) if __name__ == '__main__': # add_two_number() # print(add_two_number_1(6)) # print(two_number_2(10, 5)) # add_number(1,2, 3, 4, 5, 6) # add_number_1(1,2,3,4,5, https://jerrywang1981.github.io/post/python/06-condition-loop/ Sat, 16 May 2020 09:14:28 +0800 https://jerrywang1981.github.io/post/python/06-condition-loop/ 条件控制 #!/usr/bin/env python3 # -*- coding: utf-8 -*- if __name__ == '__main__': a = 5 if a > 0: print('a > 0') else: print('a < 0') if a > 0: pass elif a < 5: pass elif a < 10: pass else: pass if a > 0: if a < 5: print(a) else: print('a > 5') 循环控制 #!/usr/bin/env python3 # -*- coding: utf-8 -*- if __name__ == '__main__': # while condition a = 5 while a > 0: # print(a) a = a -1 # for in for i in range(10): print(i) # break the loop print('------------------') for i in range(10): print(i) else: # print('done') # continue the loop print('------------------') for i in range(10): if i == 5: continue print(i) count = 0 while count < 5: print (count, " 小于 5") count https://jerrywang1981.github.io/post/node/angular-javascript-heap-out-of-memory/ Thu, 14 May 2020 21:20:50 +0800 https://jerrywang1981.github.io/post/node/angular-javascript-heap-out-of-memory/ 如果启动angular的时候有报 Javascript heap out of memory错误，可以试着修改 node_modules/.bin/ng 加上 --max_old_space_size #!/usr/bin/env node --max_old_space_size=4096 'use strict'; https://jerrywang1981.github.io/post/go/golang-export-csv/ Tue, 12 May 2020 22:26:51 +0800 https://jerrywang1981.github.io/post/go/golang-export-csv/ 问题提出 golang最近导出csv的时候，如果用excel打开，会有乱码。在网上查到了解决方案，记录一下。 解决方案 f, err := os.Create("data.csv") if err != nil { panic(err) } defer f.Close() f.WriteString("\xEF\xBB\xBF") // 写入UTF-8 BOM，避免使用Microsoft Excel打开乱码 writer := csv.NewWriter(f) writer.Write([]string{"col 1", "col 2", "col 3"}) writer.Flush() https://jerrywang1981.github.io/post/go/convert-pdf-to-images/ Mon, 11 May 2020 21:51:52 +0800 https://jerrywang1981.github.io/post/go/convert-pdf-to-images/ 背景 在做文字检测和文字识别的时候，有时候客户提供的是pdf格式的文件，而不是jpg/png格式，这时候就需要把pdf里面多个页面 保存成图片。 代码库pdf2image 实现 Python 参考readme docker 参考readme golang 参考readme 也可以 直接下载可执行文件直接执行 -> https://github.com/jerrywang1981/pdf2images/blob/master/pdf2image https://jerrywang1981.github.io/post/go/python-go-rpc/ Mon, 04 May 2020 00:11:12 +0800 https://jerrywang1981.github.io/post/go/python-go-rpc/ 背景 有时候，我们在python中实现了一个功能，这功能如果用golang重新写呢，会比较麻烦，如果要在golang中调用python中的功能。 方式有很多，主要就是两个程序如果沟通的问题，那方式就各种各样了，可以通过http协议，json/xml等格式，或者tcp, 当然还有个 选择就 https://jerrywang1981.github.io/post/python/class-object-method/ Sat, 02 May 2020 21:59:33 +0800 https://jerrywang1981.github.io/post/python/class-object-method/ 对象中通过类或者实例的方法调用方式的不同 如果通过类中取得的方法，需要传入实例 class A(object): def test(self, *args): print(*args) if __name__ == '__main__': a = A() a.test(7, 8, 9) # method from class object m1 = A.test m1(a, 7, 8, 9, 10) # method from instance object m2 = a.test m2(7, 8, 9, 10) 也就是说, 实例方法的第一个参数是实例本身，如果写decorator的时候，要特别注意 比如print_args这个decor https://jerrywang1981.github.io/post/go/db2-backup-tool/ Sat, 25 Apr 2020 20:28:35 +0800 https://jerrywang1981.github.io/post/go/db2-backup-tool/ 为什么要做这个小工具 有时候，我们可能要把数据库一些表的数据导出，然后load到另外一个数据库，虽然有很多的工具可以选择， 但是可能用的不太顺手。所以就想，是不是能够自动生成export.sql文件。基本上来说，就是通过table，查询到 表结构，把所有的列选择出来，export到de https://jerrywang1981.github.io/post/python/05-Python-Dictionary/ Sat, 25 Apr 2020 18:35:30 +0800 https://jerrywang1981.github.io/post/python/05-Python-Dictionary/ Dictionary ###创建方式 a = dict() a = {} a = {'a': 1, 'b': 2} a = dict(a=1,b=2) a = [('a', 1), ('b', 2)] b = dict(a) a = zip('abc', [1,2,3]) a = { x:x*2 for x in range(10) } a = dict.fromkeys(range(3), 'y') https://jerrywang1981.github.io/post/python/04-python-tuple/ Fri, 24 Apr 2020 21:54:58 +0800 https://jerrywang1981.github.io/post/python/04-python-tuple/ Tuple Python 的元组与列表类似，不同之处在于元组的元素不能修改。 不可变 空元组, 一个元素元组 元组运算 元组在参数中的应用 创建tuple的方式 https://jerrywang1981.github.io/post/python/03-python-list/ Fri, 24 Apr 2020 21:05:59 +0800 https://jerrywang1981.github.io/post/python/03-python-list/ 列表 序列是Python中最基本的数据结构。序列中的每个元素都分配一个数字 - 它的位置，或索引，第一个索引是0，第二个索引是1，依此类推 索引访问 长度 空或者False 创建列表的方式 a = [] a = [1,2,3] a = list() a = list('abcdefg') a = list(range(10)) a = [ x for x in range(20) ] append In [17]: a=[1,2,3] In [18]: a.append(4) In [19]: a Out[19]: [1, 2, 3, 4] In [20]: extend In [21]: a Out[21]: [1, 2, 3, 4] In [22]: a.extend([5,6]) https://jerrywang1981.github.io/post/python/02-python-variable/ Thu, 23 Apr 2020 14:23:52 +0800 https://jerrywang1981.github.io/post/python/02-python-variable/ 变量 命名 变量名必须是大小写英文、数字和_的组合，且不能用数字开头 允许的变量名：a, a1, B_5, ____5, _C 静态语言，动态语言 大小写敏感 Dunder 规范 snake_case e.g. a_b, total_count 变量 函数名 模块 包 CamelCase e.g. Person, TeamMember 类名 常量 一般用大写, PI=3.14 https://jerrywang1981.github.io/post/python/01-python-data-type/ Thu, 23 Apr 2020 13:51:39 +0800 https://jerrywang1981.github.io/post/python/01-python-data-type/ 基本数据类型 整数 int 十进制： -1, 0, 1000, -8000, 88 八进制：0o10, 0o27 十六进制：0x10, 0xff 数学运算 + - * / // % 浮点数 float 比如：1.23, 2.6e3 字符串 str 字符串是以单引号’或双引号"括起来的任意文本 如果字符串内部包含’,", 可以用转义字符\来标识 转义 \ 不转义 https://jerrywang1981.github.io/post/go/02-prototype/ Sun, 12 Apr 2020 13:22:43 +0800 https://jerrywang1981.github.io/post/go/02-prototype/ 原型模式 参考文章 原型模式 package prototype_pattern type Person struct { Name string Age int } type Cloner interface { Clone() interface{} } func (s *Person) Clone() interface{} { var n Person n.Name = s.Name n.Age = s.Age return n } https://jerrywang1981.github.io/post/go/01-singleton/ Sun, 12 Apr 2020 12:26:51 +0800 https://jerrywang1981.github.io/post/go/01-singleton/ 单例模式 从网上找的 单例模式 golang实现 参考的文章Go语言中的单例模式 代码： 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 package example import ( "sync" ) // the singleton struct/object type singleton struct{} var instance *singleton var once sync.Once func GetSingleInstance() *singleton { once.Do(func() { instance = &singleton{} }) return instance } https://jerrywang1981.github.io/post/java/SSL-TLS-Diffie-Hellman-Modulus-1024-Bits/ Sat, 11 Apr 2020 21:17:25 +0800 https://jerrywang1981.github.io/post/java/SSL-TLS-Diffie-Hellman-Modulus-1024-Bits/ Liberty Profile 这个要修复，可能要修改两个地方。 server.xml 加入 1 2 3 <ssl id="defaultSSLConfig" keyStoreRef="defaultKeyStore" securityLevel="CUSTOM" enabledCiphers="TLS_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA256 "/> jvm.options加入 -Djdk.tls.disabledAlgorithms=SSLv3, RC4, DH keySize <768 https://jerrywang1981.github.io/post/python/python-nest/ Sat, 11 Apr 2020 21:15:14 +0800 https://jerrywang1981.github.io/post/python/python-nest/ 背景 最近有做一个nestjs的项目，用的nestjs实现的微服务，如果所有的功能都用node/nestjs写，那倒也没有什么问题了。可是有一个功能是需要用到机器学习，代码是用python写的，需要用python实现一个微服务，供nest app来调用，同时，python代码也需要调 https://jerrywang1981.github.io/post/vim/vim-online-video/ Sat, 11 Apr 2020 21:09:42 +0800 https://jerrywang1981.github.io/post/vim/vim-online-video/ 在网上发现了vim一个系列视频，是Practical Vim的作者录制的，内容很值得学习。 vim video