IT Talks 101

Pandas Data Frame vs Spark DataFrame

DataFrame ์€ ํ–‰๊ณผ ์—ด์ด ์žˆ๋Š” ๋ฐ์ดํ„ฐ ํ…Œ์ด๋ธ”์„ ๋‚˜ํƒ€๋‚ด๋ฉฐ, DataFrame ๊ฐœ๋…์€ ์–ด๋–ค ํ”„๋กœ๊ทธ๋ž˜๋ฐ ์–ธ์–ด์—์„œ๋„ ๋ณ€ํ•˜์ง€ ์•Š์ง€๋งŒ Spark ์™€ Pandas ์˜ DataFrame ์€ ์ƒ๋‹นํžˆ ๋‹ค๋ฅด๋‹ค. ์ด ๊ธ€์—์„œ๋Š” Spark DataFrame๊ณผ Pandas DataFra,e์˜ ์ฐจ์ด์ ์„ ์•Œ์•„๋ณด๋ ค๊ณ  ํ•ฉ๋‹ˆ๋‹ค. Pandas DataFrame Panda๋Š” NumPy ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ํ•˜๋Š” ์˜คํ”ˆ ์†Œ์Šค Python ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์ž…๋‹ˆ๋‹ค. ๋‹ค์–‘ํ•œ ๋ฐ์ดํ„ฐ ๊ตฌ์กฐ์™€ ์—ฐ์‚ฐ์„ ์‚ฌ์šฉํ•˜์—ฌ ์ˆ˜์น˜ ๋ฐ์ดํ„ฐ์™€ ์‹œ๊ณ„์—ด์„ ์กฐ์ž‘ํ•  ์ˆ˜ ์žˆ๋Š” Python ํŒจํ‚ค์ง€์ž…๋‹ˆ๋‹ค. ์ฃผ๋กœ ๋ฐ์ดํ„ฐ ๊ฐ€์ ธ์˜ค๊ธฐ ๋ฐ ๋ถ„์„์„ ์ƒ๋‹นํžˆ ์‰ฝ๊ฒŒ ํ•˜๊ธฐ ์œ„ํ•ด ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค. Panda DataFrame์€ ๋ ˆ์ด๋ธ”์ด ์ง€์ •๋œ ์ถ•(ํ–‰ ๋ฐ ์—ด)์„ ๊ฐ€์ง„ ์ž ์žฌ์ ์œผ๋กœ ์ด์งˆ์ ์ธ 2์ฐจ์› ํฌ๊ธฐ ๊ฐ€๋ณ€ ํ‘œ ํ˜•์‹ ..

IT Talks/BigData 2023.03.16

์•ฑ์—์„œ ์ ‘์†ํ•˜๋Š” URL ํ™•์ธํ•˜๊ธฐ

ChatGPT ์—๊ฒŒ ๋ฌผ์–ด๋ดค๋‹ค. Q) How can I capture the API URL of smartphone application ? A) To capture the API URL of a smartphone application, you will need to use a tool called a network sniffer or packet capture tool. This type of tool intercepts and captures the network traffic between your smartphone and the application's server, allowing you to inspect the requests and responses that are sent and rec..

IT Talks/AI 2023.03.03

์„œ๋ฒ„์˜ ๊ณต์ธ IP ํ™•์ธํ•˜๊ธฐ

์ผ๋ฐ˜์ ์ธ ๋ฆฌ๋ˆ…์Šค ์„œ๋ฒ„๋Š” ์ด๋”๋„ท๋ณ„๋กœ IP๋ฅผ ํ• ๋‹นํ•˜์—ฌ ifconfig ๋ช…๋ น์–ด๋กœ ํ• ๋‹น๋œ IP ๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ์œผ๋‚˜ AWS (Amazon Web Service)๋“ฑ์˜ ํด๋ผ์šฐ๋“œ ์žฅ๋น„์—์„œ๋Š” Private IP ๋งŒ ํ™•์ธ์ด ๊ฐ€๋Šฅํ•˜๊ณ  Public IP ๋Š” Amazon Console ๋“ฑ์„ ํ†ตํ•ด์„œ ํ™•์ธ์ด ๊ฐ€๋Šฅํ•˜๋‹ค. ๋˜๋Š” NAT ์•„๋ž˜์— ์œ„์น˜ํ•œ ์„œ๋ฒ„๋“ค์˜ ๊ฒฝ์šฐ๋„ ์‹ค์ œ ํ• ๋‹น๋œ IP๋Š” ์‚ฌ์„ค IP๋งŒ ํ‘œ์‹œ๋˜๊ณ , ์™ธ๋ถ€ ์ ‘์†์‹œ ์™ธ๋ถ€ ์„œ๋ฒ„์—์„œ ์ธ์‹ํ•˜๋Š” IP ๋Š” NAT ์˜ IP ๊ฐ€ ๋œ๋‹ค. ์ด๋Ÿฐ ๊ฒฝ์šฐ ์„œ๋ฒ„์—์„œ ๊ฐ„๋‹จํ•˜๊ฒŒ ์™ธ๋ถ€๋กœ ์ ‘์†ํ•˜๋Š” ๋ณธ์ธ IP๋ฅผ ํ™•์ธํ•˜๋Š” ๋ฐฉ๋ฒ•์€ ์•„๋ž˜์™€ ๊ฐ™๋‹ค. 1. ํ˜„์žฌ ์‚ฌ์šฉ์ค‘์ธ ์„œ๋ฒ„์˜ ๊ณต์ธIP ์กฐํšŒ (Outbound) $ curl ip.ojj.kr Your IP is 175.123.191.3 2. ํŠน์ • IP์˜ ์ •๋ณด ์กฐํšŒ $ cu..

IT Talks/Linux 2023.02.22

[๋ฆฌ๋ˆ…์Šค] date ๋ช…๋ น์œผ๋กœ timestamp ๋ณ€ํ™˜ํ•˜๊ธฐ

1. timestamp ๊ฐ’์œผ๋กœ ์›ํ•˜๋Š” ๋‚ ์งœ ํ˜•ํƒœ๋กœ ๋ณ€ํ™˜ํ•˜๊ธฐ $ date -d '1970-01-01 1676360315 sec' 2023. 02. 14. (ํ™”) 07:38:35 KST $ date -d '1970-01-01 1676360315 sec' +'%Y-%m-%d %H:%M:%S' 2023-02-14 07:38:35 * ๊ฒฐ๊ณผ๋Š” ์‹œ์Šคํ…œ Timezone์— ๋งž์ถฐ์„œ ํ‘œ์‹œ๋จ 2. ๋‚ ์งœ ํ˜•์‹์œผ๋กœ timestamp ๊ฐ’ ๊ตฌํ•˜๊ธฐ (1๋ฒˆ๊ณผ ๋ฐ˜๋Œ€) $ date -d '2022-02-14' +%s 1644764400

IT Talks 2023.02.14

ChatGPT ๊ฐ€ ์ถ”์ฒœํ•˜๋Š” ๊ฐœ๋ฐœ์ž ์ปค๋ฆฌ์–ด

Q. Please recommend a career as a software developer. A. A career as a software developer can be a very rewarding and fulfilling choice. Here are some reasons why: 1. High demand: Software development is a rapidly growing field with a high demand for skilled professionals. This means that there are many job opportunities available and the demand for software developers is expected to continue to..

IT Talks/AI 2023.02.13

๋น„๋ฐ€๋ฒˆํ˜ธ ์ž…๋ ฅ ์—†์ด MySQL ์ ‘์†ํ•˜๊ธฐ

MySQL ๋ฒ„์ „์ด ์ ์  ์˜ฌ๋ผ๊ฐ€๋ฉด์„œ ์–ธ์ œ๋ถ€ํ„ฐ์ธ๊ฐ€ mysql ๋ช…๋ น์–ด๋กœ ์ ‘์†์‹œ ๋น„๋ฐ€๋ฒˆํ˜ธ๋ฅผ ๊ฐ™์ด ์ž…๋ ฅํ•˜๋ฉด ์•„๋ž˜์™€ ๊ฐ™์ด ์•ˆ์ „ํ•˜์ง€ ์•Š๋‹ค๋Š” ๊ฒฝ๊ณ ๊ฐ€ ๋‚˜์˜จ๋‹ค. $ mysql -hlocalhost -udb_user -pdb_password mysql: [Warning] Using a password on the command line interface can be insecure. ์‰˜ ์Šคํฌ๋ฆฝํŠธ๋กœ DB ์ ‘์†ํ•˜๋Š” ์ž‘์—…์„ ํ•˜๋Š” ๊ฒฝ์šฐ ์œ„์™€ ๊ฐ™์ด ๋ช…๋ น์–ด์— ๋น„๋ฐ€๋ฒˆํ˜ธ๊นŒ์ง€ ๊ฐ™์ด ์ž…๋ ฅํ•˜๋Š” ๊ฒฝ์šฐ๊ฐ€ ๋งŽ์€๋ฐ ์‹คํ–‰์€ ๋˜์ง€๋งŒ ๊ฒฝ๊ณ ๊ฐ€ ๋‚˜์˜ค๋Š”๊ฒŒ ๋งค์šฐ ๊ฑฐ์Šฌ๋ฆฐ๋‹ค. ์ด๋Ÿฐ ๊ฒฝ์šฐ๋Š” ์•„๋ž˜ 2๊ฐ€์ง€ ๋ฐฉ๋ฒ•์œผ๋กœ ํ•ด๊ฒฐ์ด ๊ฐ€๋Šฅํ•˜๋‹ค. ๋ฐฉ๋ฒ•1) ๊ณ„์ • home path (๋ฆฌ๋ˆ…์Šค ๊ธฐ์ค€ /home/userid ๋“ฑ) ์— .my.cnf ํŒŒ์ผ์„ ์ƒ์„ฑํ•˜๊ณ  ์•„๋ž˜์™€ ๊ฐ™์ด ๋‚ด์šฉ์„ ์ž…๋ ฅํ•œ๋‹ค. ..

IT Talks/Database 2023.02.09

Data Scientist vs Engineer vs Analyst

์˜ˆ์ „์— ์ธํ„ฐ๋„ท์—์„œ ๋ดค๋˜ ์ด๋ฏธ์ง€ ์ธ๋ฐ, ๋ฐ์ดํ„ฐ ๊ด€๋ จ ์ง๋ฌด์— ๋Œ€ํ•ด ์ •๋ฆฌ๊ฐ€ ์ž˜ ์•ˆ๋˜๋Š” ๊ฒฝ์šฐ๊ฐ€ ๋งŽ์•„์„œ ์ฐธ๊ณ ํ•˜๋ฉด ์ข‹์„ ๊ฒƒ ๊ฐ™๋‹ค. (ํ˜„์‹ค์€ ๊ทธ๋ ‡์ง€ ์•Š์ง€๋งŒ) Data Scientist ํ†ต๊ณ„๋‚˜ ๋จธ์‹  ๋Ÿฌ๋‹์„ ์ด์šฉํ•ด ์ฃผ์š” ๋น„์ฆˆ๋‹ˆ์Šค ์งˆ๋ฌธ์— ๋Œ€ํ•œ ์˜ˆ์ธก๊ณผ ๋‹ต๋ณ€์„ ๋งŒ๋“ฆ. Data Engineer ๋ฐ์ดํ„ฐ ๊ณผํ•™์ž์™€ ๋ถ„์„๊ฐ€๊ฐ€ ์ž‘์—…์„ ์ˆ˜ํ–‰ํ•  ์ˆ˜ ์žˆ๋Š” ์‹œ์Šคํ…œ์„ ๊ตฌ์ถ•ํ•˜๊ณ  ์ตœ์ ํ™”. Data Analyst ๋ฐ์ดํ„ฐ๋ฅผ ํ™œ์šฉํ•˜์—ฌ ๋น„์ฆˆ๋‹ˆ์Šค ์˜์‚ฌ ๊ฒฐ์ •์— ๋„์›€์ด ๋˜๋Š” ๊ฒฐ๊ณผ๋ฅผ ์ „๋‹ฌํ•จ์œผ๋กœ์จ ๊ฐ€์น˜๋ฅผ ์ œ๊ณต. (์ฐธ๊ณ ) ๋ฐ์ดํ„ฐ ์ง๋ฌด๋ณ„ ์Šคํ‚ฌ ๋ฐ์ดํ„ฐ ์ง๋ฌด๋ณ„ ์Šคํ‚ฌ - Types of Data Professionals ์ „์— ๋ฐ์ดํ„ฐ ์ง๋ฌด๋ณ„ ํ•˜๋Š”์ผ์— ๋Œ€ํ•œ ํฌ์ŠคํŒ…์„ ์˜ฌ๋ฆฐ ์ ์ด ์žˆ์—ˆ๋Š”๋ฐ (์•„๋ž˜ ๊ธ€ ์ฐธ๊ณ ) ์ด๋ฒˆ์—๋Š” ๊ฐ ๋ฐ์ดํ„ฐ ์ง๋ฌด๋ณ„ ์ฃผ์š” ์—…๋ฌด์— ๋Œ€ํ•ด ์ฐจํŠธ ํ˜•ํƒœ๋กœ ๋œ ๊ทธ๋ฆผ์ด ์žˆ์–ด์„œ..

IT Talks/BigData 2022.10.06

๋„ค์ด๋ฒ„ ๋ธ”๋กœ๊ทธ ์ฃผ์†Œ ๋ณ€๊ฒฝ ๊ธฐ๋Šฅ ์ถœ์‹œ

๋„ค์ด๋ฒ„ ๋ธ”๋กœ๊ทธ๋Š” ์„œ๋น„์Šค ์ถœ์‹œ๋•Œ ๋ถ€ํ„ฐ ๋„ค์ด๋ฒ„ID๋ฅผ ๊ทธ๋Œ€๋กœ ์‚ฌ์šฉํ•ด์•ผ ํ•˜๋Š” ๋ฐฉ์‹์ด์—ฌ์„œ ์ŠคํŒธ ๋ฉ”์ผ์ด๋‚˜ ๊ฐœ์ธ์ •๋ณด ๋…ธ์ถœ ๊ด€๋ จ ์ด์Šˆ๊ฐ€ ๊ณ„์† ์žˆ์—ˆ๋‹ค. ๋น„์Šทํ•œ ์ด์Šˆ๋กœ, ๋„ค์ด๋ฒ„์นดํŽ˜ ํšŒ์› ๋ชฉ๋ก์—์„œ ๋„ค์ด๋ฒ„ID๊ฐ€ ๋…ธ์ถœ๋˜์–ด, ํšŒ์› ๋ชฉ๋ก์„ ์ˆ˜์ง‘ํ•  ์ˆ˜ ์žˆ์—ˆ์œผ๋‚˜ ID ๋ถ€๋ถ„์„ ์•”ํ˜ธํ™” ํ•˜์—ฌ ํšŒํ”ผ(?)ํ•˜๋Š” ๋ฐฉ์‹์„ ์ ์šฉํ–ˆ์—ˆ๋Š”๋ฐ, ์˜ค๋Š˜ ๋„ค์ด๋ฒ„ ๋ธ”๋กœ๊ทธ ๊ณต์ง€ ๋ฉ”์ผ์„ ๋ณด๋‹ˆ ๋ธ”๋กœ๊ทธ์—์„œ ์ฃผ์†Œ๋กœ ์‚ฌ์šฉ๋˜๋Š” ๋„ค์ด๋ฒ„ID ๋ถ€๋ถ„์„ ๋ณ€๊ฒฝํ•˜๋Š” ๊ธฐ๋Šฅ์ด ์ถœ์‹œ๋œ๋‹ค๋Š” ๋‚ด์šฉ์ด์—ˆ๋‹ค. 10์›”12์ผ๋ถ€ํ„ฐ ๋ณ€๊ฒฝ ๊ฐ€๋Šฅํ•˜๋‹ค๋‹ˆ ๊ธฐ์กด์— ๋„ค์ด๋ฒ„ID๊ฐ€ ๋ธ”๋กœ๊ทธ ์ฃผ์†Œ์— ๊ทธ๋Œ€๋กœ ๋…ธ์ถœ๋˜๋Š”๊ฑธ ๊บผ๋ฆฌ์…จ๋˜ ๋ถ„๋“ค์€ ์ด๋ฒˆ์— ๋ฐ”๊ฟ”๋ณด์‹œ๋Š”๊ฒƒ๋„ ์ข‹์„ ๊ฒƒ ๊ฐ™๋‹ค. ๋„ค์ด๋ฒ„๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ํšŒ์›์ด ๋งŽ๋‹ค ๋ณด๋‹ˆ ๊ฐœ์ธ์ •๋ณด ๊ด€๋ จ ์ด์Šˆ๋„ ๋งŽ์•„์„œ ์ด์™€ ๊ด€๋ จ๋œ ๊ฐœ์„  ์ž‘์—…์„ ๊ณ„์† ํ•˜๋Š”๊ฒƒ ๊ฐ™๊ธด ํ•œ๋ฐ ์ข€ ์˜ค๋ž˜ ๊ฑธ๋ฆฌ๋Š”๊ฒƒ ๊ฐ™๊ธฐ๋„ ^^ ๋ธ”๋กœ๊ทธ์—์„œ ์•Œ๋ ค๋“œ..

IT Talks 2022.09.30

Python3 ๊ฐ€์ƒ ํ™˜๊ฒฝ ์ƒ์„ฑ

venv ๋ชจ๋“ˆ์€ ์ž์ฒด ์‚ฌ์ดํŠธ ๋””๋ ‰ํ„ฐ๋ฆฌ๋ฅผ ๊ฐ–๋Š” ๊ฒฝ๋Ÿ‰ โ€œ๊ฐ€์ƒ ํ™˜๊ฒฝโ€์„ ๋งŒ๋“ค๊ณ , ์„ ํƒ์ ์œผ๋กœ ์‹œ์Šคํ…œ ์‚ฌ์ดํŠธ ๋””๋ ‰ํ„ฐ๋ฆฌ์—์„œ ๊ฒฉ๋ฆฌํ•  ์ˆ˜ ์žˆ๋„๋ก ์ง€์›ํ•ฉ๋‹ˆ๋‹ค. ๊ฐ ๊ฐ€์ƒ ํ™˜๊ฒฝ์€ ๊ณ ์œ ํ•œ ํŒŒ์ด์ฌ ๋ฐ”์ด๋„ˆ๋ฆฌ(์ด ํ™˜๊ฒฝ์„ ๋งŒ๋“œ๋Š” ๋ฐ ์‚ฌ์šฉ๋œ ๋ฐ”์ด๋„ˆ๋ฆฌ ๋ฒ„์ „๊ณผ ์ผ์น˜ํ•จ)๋ฅผ ๊ฐ€์ง€๋ฉฐ ์ž์‹ ์˜ ์‚ฌ์ดํŠธ ๋””๋ ‰ํ„ฐ๋ฆฌ์— ๋…๋ฆฝ์ ์œผ๋กœ ์„ค์น˜๋œ ํŒŒ์ด์ฌ ํŒจํ‚ค์ง€ ์ง‘ํ•ฉ์„ ๊ฐ€์งˆ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๊ฐ€์ƒํ™˜๊ฒฝ ์ƒ์„ฑ $ python3 -m venv /home/myhome/venv * ์ง€์ •ํ•œ ๋””๋ ‰ํ† ๋ฆฌ์— ๊ฐ€์ƒ ํ™˜๊ฒฝ ์ƒ์„ฑ (๋ฆฌ๋ˆ…์Šค ๊ธฐ์ค€) * ์‹คํ–‰ ํ›„ ์ด๋ž˜์™€ ๊ฐ™์€ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•œ๋‹ค๋ฉด --without-pip ์˜ต์…˜์œผ๋กœ pip ๋Š” ๋ณ„๋„๋กœ ์„ค์น˜ํ•ด ์ค€๋‹ค. Error: Command '['/home/ojj/venv/bin/python3', '-Im', 'ensurepip', '--upgrade',..

IT Talks 2022.09.22

๋งฅ(Mac)์—์„œ ์นด์นด์˜คํ†ก ์•ฑ 2๊ฐœ ์‚ฌ์šฉํ•˜๊ธฐ

์นด์นด์˜คํ†ก์„ ํ•„์š”์— ๋”ฐ๋ผ 2๊ฐœ๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๋ถ„๋“ค๋„ ๋งŽ์ด ์žˆ์„ํ…๋ฐ, ์•ˆ๋“œ๋กœ์ด๋“œํฐ์˜ ๊ฒฝ์šฐ์—๋Š” ๋“€์–ผ ๋ฉ”์‹ ์ € ๊ธฐ๋Šฅ์œผ๋กœ ์นด์นด์˜คํ†ก, ๋ผ์ธ, ํŽ˜์ด์Šค๋ถ๋ฉ”์‹ ์ € ๋“ฑ๋“ฑ์„ 2๊ฐœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋‹ค. (์•ˆ๋“œ๋กœ์ด๋“œ 7.0 ๋ถ€ํ„ฐ ์ƒ๊ฒผ๋‹ค๊ณ  ํ•จ) ์• ํ”Œ PC (๋งฅ, Mac)์—์„œ๋„ ์นด์นด์˜คํ†ก์•ฑ์„ ์„ค์น˜ํ•ด์„œ ์‚ฌ์šฉํ•˜๊ณ  ์žˆ๋Š”๋ฐ, ๋ชจ๋ฐ”์ผ์— ์žˆ๋Š” 2๊ฐœ์˜ ์นดํ†ก์„ PC์—์„œ๋„ ๋™์‹œ์— ์‹คํ–‰์ด ๊ฐ€๋Šฅํ• ์ง€ ์ฐพ์•„๋ณด๋‹ค๊ฐ€ ์ฐพ์€ ๋ฐฉ๋ฒ•์œผ๋กœ ๊ฐ„๋‹จํžˆ ์Šคํฌ๋ฆฝํŠธ๋ฅผ ๋งŒ๋“ค์—ˆ๋‹ค. App Store ์—์„œ ์นด์นด์˜คํ†ก์„ ์„ค์น˜ํ•œ ํ›„์— ์•„๋ž˜ ์Šคํฌ๋ฆฝํŠธ๋ฅผ ์‹คํ–‰ํ•˜๋ฉด, KakaoTalk2 ๋ผ๋Š” ์•ฑ์œผ๋กœ ๋ณต์‚ฌํ•ด์„œ 2๊ฐœ๋ฅผ ๋”ฐ๋กœ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ๋„๋ก ํ•œ๋‹ค. * ์†Œ์Šค : https://github.com/okojj/utils/blob/main/kakao-talk/copy-kakaotalk-app-mac.sh #!/bin..

IT Talks 2022.09.06
728x90