端端舝

手手切鈭TECH市白尼
Kinect匹AR諾嶲卞⻌曰仇手丹

赻撩畿賡
庌靡ㄩ憚蚗喟(Takashi Yoshinaga)
垀扽ㄩ嬝笣珂傷褪悝撮胍旃噶垀(ISIT)
�嬡ㄩAR/VR毛蚚中凶ロ�褫�趙午跪意�䛐
戊立亙瓦氾奴ㄩAR戊件氾件汁釬傖辭�頗翋殼

AR戊件氾件汁釬傖辭�頗及畿賡
? 2013爛5堎卞辭�頗毛旦正奈玄
? AR戊件氾件汁及釬曰源毛甩件朮左件倛宒匹悝少
? �杅反5~10靡最僅及屾�杅匹灍囥
? 統樓沭璃反AR/VR卞鶸庤互丐月�(眭舑祥猁)
? 跪華匹嶱殼 (腦𠈹﹜倱掛﹜龧ぃ﹜墿ぃ﹜湮煦﹜ 繒�憷﹜刓諳﹜𣷣憷﹜憝𡌂)

Twitter午辭�頗矢奈斥匹ロ�毛逃陓仄化中引允
#AR_Fukuoka Google匹☆AR辭�頗★匹𨈘坰

本永玄失永皿
? Kinect for Windows SDK v1.8
https://www.microsoft.com/en-us/download/details.aspx?id=40278
? Processing 2.2.1
https://processing.org/download/
Processing毛お�摽﹜狟�毛賤�仄凶白巧伙母毛
Processing及libraries白巧伙母卞饜离
(Windows及�磁ㄩC:?Users?YOU?Documents?Processing? libraries)
? SimpleOpenNI 1.96
https://code.google.com/archive/p/simple-openni/downloads
? NyARToolKit (nyar4psg.zip)
https://github.com/nyatla/NyARToolkit-for-Processing/releases
狟�毛賤�仄化犯旦弁玄永皿脹及歹井曰支允中午仇欠卞饜离
https://drive.google.com/file/d/1ahzAzp_s5gqg5WbfJkF5JSv5v87qm
KZv/view?usp=sharing

掛゜釬傖允月戊件氾件汁
Kinect匹龰腕仄凶萸�犯奈正(Point Cloud)毛伉失伙正奶丞卞AR桶尨
https://youtu.be/WeorvV9vZh8

Kinect午反
? Kinectㄗ平生弁玄ㄘ午反Microsoft扦及必奈丞辻﹜XBOX360
�蚚及斥尼旦民乓奈⻌薯戊件玄伕奈仿午仄化逃仫今木凶辻け
? 市仿奈賒砉?旮僅(Depth)ロ�?旯极及庲舑脹互褫夔
旮僅本件扔(喪俋�皿伕斥尼弁正ㄚ喪俋�市丟仿)
市仿奈市丟仿

Kinect毛蚚中凶戊件氾件汁嶱逃
∥ 市仿奈賒砉+擒褩賒砉+3D�䛐
�极庲舑 ↙
午C# or C++毛妏尹壬嶱逃褫夔
磐�痱懈互詢中毛妏丹午掀廌腔��g卞嶱逃褫夔

processing午反
? processing午反﹜�赽失奈玄午申斥亙失伙犯扒奶件及凶戶及
皿伕弘仿立件弘晟惤匹丐曰﹜緙磁嶱逃虐噫
? ��腔卅桶尨毛��g卞俴尹月凶戶場陑氪及皿伕弘仿立件弘
悝�卞手砃中化中月﹝恅楊反Java午幻廿肮元﹝

ARToolKit﹜NyARToolKit
? AR互珨啜卞庲眭今木月五勻井仃午卅勻凶仿奶皮仿伉
? 市丟仿午淏源倛及穴奈市奈毛妏蚚仄化CG毛笭�
? 手午手午反C/C++砃仃卞嶱逃
? processing�𡛟及巖汜唳仿奶皮仿伉NyARToolKit
賒砉及龰腕 ? 媼�趙脹及賒砉�I燴
? 穴奈市及𨈘堤
弇离?訬��呾 CG毛笭�
(竘蚚啋:
http://www.youtube.com/watch?v=TqGAqAFlGg0)
(D.Wagner et al. Computer Vision Winter
Workshop, 2007)

𨃨�
Kinect午PC毛諉適仄化云五引仄斤丹

processing毛お�
灍俴
氾平旦玄巨犯奴正
仇仇卞末奈旦戊奈玉毛�扴
悵湔
▽お�▼
? processing
及失奶戊件毛
弁伉永弁
▽悵湔▼
? 悵湔示正件毛
弁伉永弁
? Sample1午
仄化悵湔

processing匹它奴件玉它毛桶尨今六方丹
//setup憝杅囀匹場ヽ偞隅
void setup(){
size(256,256); //它奴件玉它囀窒及扔奶朮
background(200,200,200); //它奴件玉它囀窒及伎
}
犯白巧伙玄匹反伎毛RGB匹桶政(公木冗木0~255)

價掛�倛毛鏡賒仄方丹
void setup(){
/*吽謹(ゴ矢奈斥統桽)*/
}
//鏡賒及凶戶及憝杅
void draw(){
//��及伎
stroke(255,0,0);
//��及怮今(pixel)
strokeWeight(10);
//凤曰勾少仄及伎
fill(0,255,255);
//侐褒倛(rectangle)毛鏡賒
//rect(x釱㻢,y釱㻢,筵盟,磬盟)
rect(10,10,50,50);
}
(x,y)=(10,10)
��及伎ㄩ(255,0,0)=喪
��及怮今ㄩ10px
凤曰伎ㄩ
(0,255,255)=扑失件

坻及�倛手鏡中化心方丹
void draw(){
//��及伎
stroke(255,0,0);
//��及怮今(pixel)
strokeWeight(10);
//凤曰勾少仄及伎
fill(0,255,255);
//侐褒倛(rectangle)毛鏡賒
//rect(x釱㻢,y釱㻢,筵盟,磬盟)
rect(10,10,50,50);
}
凤曰勾少仄凶�毛鏡賒
赻蚕卅嗣褒倛(ʊ褒倛)毛鏡賒

�萸毛硌隅仄化�倛毛鏡賒
▽ʊ褒倛▼
strokeWeight(2); //��及盟
beginShape(TRIANGLES);
vertex(70, 10);
vertex(70, 60);
vertex(150, 60);
endShape();
(70,10)
(70,60) (150,60)
▽萸(凤曰勾少仄�)▼
strokeWeight(60); //萸及盟(=�及眻噤)
beginShape(POINTS);
vertex(35,100); //萸及笢陑弇离(x,y)
endShape(); 60
60
(35,100)
�� https://processing.org/reference/beginShape_.html

棒及皿伕斥尼弁玄及釬傖
悵湔陔�釬傖 ▽陔�釬傖▼
? 陔�示正件毛弁伉永弁
? Sample1及囀⺍毛
戊疋奈ㄕ矢奈旦玄
? draw憝杅囀毛秏允
▽悵湔▼
? 悵湔示正件毛弁伉永弁
? Sample2午仄化悵湔

void draw(){
stroke(255,0,0); //喪匹凤曰勾少仄
strokeWeight(30); //眻噤毛本永玄
beginShape(POINTS);
for(int x=0; x<5; x+=1){
vertex(40*x, 0); //�
}
endShape();
}
for恅毛蚚中凶靜曰殿仄�I燴 (1/2)
x=0 x=1 x=2 x=3 x=4
40 (=40*1)
80 (=40*2)

for恅毛蚚中凶靜曰殿仄�I燴 (2/2)
void draw(){
stroke(255,0,0); //喪匹凤曰勾少仄
strokeWeight(30); //眻噤毛本永玄
beginShape(POINTS);
for(int y=0; y<5; y+=1){
for(int x=0; x<5; x+=1){
vertex(40*x, 0); //�
}
}
endShape();
}
vertex(40*x, 40*y); y ↙ 0
y ↙ 1
y ↙ 2
y ↙ 3
y ↙ 4

鏡賒弇离毛內日公丹
void draw(){
translate(20,20); //�极毛x,y源砃卞20內勾痄�
stroke(255,0,0);
strokeWeight(30);
beginShape(POINTS);
for(int y=0; y<5; y+=1){
for(int x=0; x<5; x+=1){
vertex(40*x, 40*y);
}
}
endShape();
}

void draw(){
translate(20,20); //�极毛x,y源砃卞20內勾痄�
stroke(255,0,0);
strokeWeight(30);
beginShape(POINTS);
for(int y=0; y<5; y+=1){
for(int x=0; x<5; x+=1){
vertex(40*x, 40*y);
}
}
endShape();
}
1�云五卞鏡賒仄方丹
for(int y=0; y<5; y+=1){
for(int x=0; x<5; x+=2){
vertex(40*x, 40*y);
}
}
for(int y=0; y<5; y+=2){
for(int x=0; x<5; x+=2){
vertex(40*x, 40*y);
}
}

棒及皿伕斥尼弁玄及釬傖
悵湔陔�釬傖 ▽陔�釬傖▼
? 陔�示正件毛弁伉永弁
? Sample2及囀⺍毛
戊疋奈ㄕ矢奈旦玄
? setup憝杅午draw憝杅
及囀�及戊奈玉毛秏允
▽悵湔▼
? 悵湔示正件毛弁伉永弁
? Sample3午仄化悵湔

Kinect毛お�今六方丹
import SimpleOpenNI.*; //SimpleOpenNI毛掂心煋戈
SimpleOpenNI kinect; //Kinect毛㜳丹凶戶及劐杅
void setup(){
//OpenNI云方太Kinect毛場ヽ趙
kinect = new SimpleOpenNI(this);
if(kinect.isInit() == false){
exit();
return;
}
}

復庲
喪俋�及桽扞互宎引月�磁手
巨仿奈互堤卅仃木壬OK

NyARToolKit毛妏丹𨃨�
岈ゴ卞DL仄凶Sample白巧伙母囀及camera_para.dat,
patt.sample1毛Sample3白巧伙母卞戊疋奈ㄕ矢奈旦玄
▽NyARToolKit▼ ▽Sample3▼

ARToolKit匹賒砉毛桶尨允月𨃨�
import SimpleOpenNI.*;
import jp.nyatla.nyar4psg.*; //仿奶皮仿伉及奶件禾奈玄
SimpleOpenNI kinect;
MultiMarker ar; //ARToolKit及辻夔毛妏丹凶戶及劐杅
void setup(){
size(640,480,P3D);
/*OpenNI云方太Kinect毛場ヽ趙(吽謹)*/
ar = new MultiMarker(
this, width, height, "camera_para.dat"
,NyAR4PsgConfig.CONFIG_PSG_PV221);
}
640℅480及薆郖卞
3D及鏡賒毛俴丹
市丟仿及賒褒卞憝允月ロ�賒砉扔奶朮

ARToolKit匹市丟仿賒砉毛桶尨允月
void draw(){
//Kinect及龰腕ロ�毛載陔
kinect.update();
//市仿奈賒砉毛龰腕
PImage rgbImage = kinect.rgbImage();
//ARToolKit枑鼎及辻夔匹賒砉毛鏡賒允月
ar.drawBackground(rgbImage);
}
void setup(){
ar = new MultiMarker(this, width, /*眕狟吽謹*/);
kinect.enableRGB(); //市仿奈賒砉及龰腕毛ON
}

ARToolKit匹穴奈市奈毛庲舑仄方丹 (1/2)
void setup(){
ar = new MultiMarker(this, width, /*眕狟吽謹*/);
//穴奈市奈及犯奈正毛掂心煋戈(扔奶朮反80mm)
ar.addARMarker("patt.sample1", 80);
kinect.enableRGB();
}
▽ARToolKit穴奈市奈及價渙眭舑▼
? 淏源倛及�午囀�及由正奈件
? 隙��備由正奈件反NG
? ARToolKit(掛模)毛母它件伕奈玉
允木壬杅意�及穴奈市互瞳蚚褫夔
? 穴奈市奈犯奈正及赻釬手褫夔

ARToolKit匹穴奈市奈毛庲舑仄方丹 (2/2)
void draw(){
kinect.update();
}
ar.detect(rgbImage); //穴奈市奈毛𨈘堤
if(!ar.isExistMarker(0)){
return; //穴奈市奈互湔婓仄卅仃木壬𥽋月
}
ar.beginTransform(0); //穴奈市0毛埻萸卞仄化鏡賒
fill(0,0,255);
box(40); //磬筵詢今互40mm及蕾源极
ar.endTransform();

�釬復庲
穴奈市奈卞�\勻化
中月覜元互仄卅中

CG及弇离毛捼淕仄方丹
ar.beginTransform(0);
translate(0,0,20);
fill(0,0,255);
box(40);
ar.endTransform();
?box毛奻卞圉煦內日允
x
Y
Z
(0,0,0)
(0,0,20)
40mm

擒褩(Depth)賒砉毛龰腕仄方丹
void setup(){
/*吽謹(Kinect及場ヽ偞隅卅升)*/
kinect.enableRGB();
kinect.enableDepth(); //擒褩賒砉及瞳蚚毛ON
}
void draw(){
kinect.update();
//PImage rgbImage = kinect.rgbImage();
PImage rgbImage = kinect.depthImage();
/*眕狟吽謹*/
}

�釬復庲
棒反仇木毛3棒啋趙允月
Kinect井日及擒褩卞
𡛟元凶吲筏

公及ゴ卞???
void setup(){
/*吽謹(Kinect及場ヽ偞隅卅升)*/
kinect.enableRGB();
kinect.enableDepth(); //擒褩賒砉及瞳蚚毛ON
}
void draw(){
kinect.update();
//PImage rgbImage = kinect.depthImage();
/*眕狟吽謹*/
}
桶尨支穴奈市奈庲舑卞妏蚚允月賒砉毛市仿奈卞𥽋允
仇及戊奈玉卞𥽋允
祅壺

擒褩賒砉毛3棒啋ロ�卞劐𡥼
void draw(){
kinect.update();
//磬筵(pixel)午Depth及瞎毛x,y,z釱㻢(mm)卞劐𡥼
//PVector及珨棒啋饜蹈卞�犯奈正毛�嶕
PVector[] realWorld = kinect.depthMapRealWorld();
/*眕狟吽謹*/
pixel午吲筏
劐𡥼
灍諾嶲

擒褩賒砉午realWorld午及�𡛟葆仃
P00
(x,y,z)
P01
(x,y,z)
P02
(x,y,z)
P10
(x,y,z)
P11
(x,y,z)
P12
(x,y,z)
P20
(x,y,z)
P21
(x,y,z)
P23
(x,y,z)
▽禾奶件玄▼
? 3棒啋釱㻢反珨棒
啋饜蹈realWorld
卞跡慮今木化中月
? ￤砩及u,v疋弁本伙
及x,y,z毛眭月卞反
�呾互斛猁
P00 P01 P02 P10 P11 P12 P20 P21 P22 ???
0 1 2
v
0
1
2
u
realWorld
[0] [1] [2] [3] [4] [5] [6] [7] [8] ???
v = 0 僇醴 v = 1 僇醴 v = 2 僇醴
疋弁本伙u,v匹及3棒啋釱㻢 ↙ realWorld[vㄙ賒砉及筵盟 + u]
▽擒褩賒砉▼

3D犯奈正毛萸匹鏡賒仄方丹 (1/2)
void draw(){
/*Kinect及賒砉龰腕支劐杅及哫晟(吽謹)*/
/*穴奈市奈及庲舑 (吽謹)*/
ar.beginTransform(0); //
translate(0,0,20);
fill(0,0,255);
box(40);
ar.endTransform();
}
beginShape(POINTS);
//仇仇匹鏡賒允月萸及3棒啋釱㻢毛硌隅 (棒�)
endShape();
translate(0,0,20);
fill(0,0,255);
box(40);

3D犯奈正毛鏡賒仄方丹 (2/2)
beginShape(POINTS);
endShape();
//v:磬, u:筵﹜公木冗木4pixel云五卞3棒啋釱㻢毛龰腕
for(int v=0; v<kinect.depthHeight(); v+=4){
for(int u=0; u<kinect.depthWidth(); u+=4){
int index = u + v * kinect.depthWidth();
PVector point = realWorld[index];
stroke(255,255,0); //鏡賒伎(酴伎)
if(point.z>0) {
vertex(point.x, point.y, point.z);
}
}
}
X
Y
Z
𧃸賒匼(z=0)反
鏡賒仄凶仁卅中

�釬復庲
睡手桶尨今木卅中ˋ
摽欠卞褩木化穴奈市奈
毛湧荌允月午酴伎中萸

政袨毛參挍仄方丹
Kinect井日杅100mm珂卞�䛐�砓互中月 Kinect匹�䛐仄凶萸毛饜离 (埻萸反穴奈市奈)
▽政皿伕弘仿丞及及�觳萸▼
穴奈市奈井日湮五仁褩木凶
弇离及萸鏡毛市丟仿(Kinect)卞
砃仃月午賒砉醱井日輪允亢月
萸鏡及弇离毛穴奈市奈埻萸
笚煘卞內日允斛猁互丐月
鏡賒

3D犯奈正毛埻萸葆輪卞痄�今六方丹
translate(0,0,-600); //600mm摽欠卞𥽋允 (�磁卞方月)
beginShape(POINTS);
PVector point = realWorld[index];
stroke(255,255,0); //鏡賒伎
if(point.z>0) {
}
}
}
endShape();
X
Y
Z

�釬復庲
萸及躇僅互屾仄詢仁卅月
穴奈市奈及煘曰卞灍渡湮及赻煦互鏡賒今木月ㄗ日仄中ㄘ

3D犯奈正毛忒�\曰扔奶朮卞仄方丹
scale(0.3); //扔奶朮毛3/10卞允月
translate(0,0,-600);
beginShape(POINTS);
PVector point = realWorld[index]; //
stroke(255,255,0); //鏡賒伎
if(point.z>0) {
}
}
}

鏡賒允月萸及湮五今毛淕尹月
strokeWeight(2); //鏡賒扔奶朮毛2pixel卞允月
scale(0.3);
beginShape(POINTS);
/*眕狟鏡賒及�I燴(吽謹)*/

穴奈市奈及奻卞蕾凶六方丹
strokeWeight(2);
scale(0.3);
//x嵞(1,0,0)毛笢陑卞90僅(=羽/2)隙�
rotate(PI/2, 1, 0, 0);
beginShape(POINTS);
隙�

掖劓及犯奈正毛壺�ㄩ鏡賒��及癹隅
beginShape(POINTS);
stroke(255,255,0);
if(point.z>0) {
}
}
}
endShape();
if(point.z>0 && point.z<1500) {
}
壺�
1500mm帤㦤卅日鏡賒

鏡賒弇离毛奻卞內日允
痄�
strokeWeight(2);
scale(0.3);
rotate(PI/2, 1, 0, 0);
translate(0,0,-600); //600mm摽欠卞𥽋允 (�磁卞方月)
beginShape(POINTS);
/*眕狟鏡賒及�I燴(吽謹)*/
translate(0,300,-600); //300mm奻卞痄� (�磁卞方月)

萸鏡卞市丟仿賒砉及伎毛賃曰絞化方丹
beginShape(POINTS);
color pixelColor = rgbImage.pixels[index];
stroke(255,255,0);
if(point.z>0) {
}
}
}
endShape();
stroke(pixelColor); //跪疋弁本伙及伎毛硌隅

�釬復庲
切斤勻午朮伊月

內木互汜元月燴蚕ㄩKinect瞳蚚及蛁砩萸
Color Depth
? 市仿奈蚚及市丟仿午兜俴蚚及
市丟仿反弇离手賒褒手�卅月
↙ 跪賒砉及疋弁本伙釱㻢
反珨祡仄卅中

市仿奈賒砉午擒褩賒砉及內木及娗淏
void setup(){
size(640, 480, P3D);
kinect = new SimpleOpenNI(this);
if(kinect.isInit() == false){
exit();
return;
}
ar = new MultiMarker(
this, width, height, "camera_para.dat");
ar.addARMarker("patt.sample1", 80);
kinect.enableRGB();
kinect.enableDepth();
kinect.alternativeViewPointDepthToImage();
}
仇木毛袚樓允月分仃

▽統蕉▼
? processing伉白央伊件旦
https://processing.org/reference/
? SimpleOpneNI鼠宒扔奶玄
https://code.google.com/p/simple-openni/
? NyARToolKit鼠宒扔奶玄
http://nyatla.jp/nyartoolkit/wp/
? ARToolKit鼠宒扔奶玄(郔陔)
http://artoolkit.org/

鬼勳紳梗釵喧匹插賊諾潔卞⻌曰煋手丹

端端舝

鬼勳紳梗釵喧匹插賊諾潔卞⻌曰煋手丹

Recommended

More Related Content

What's hot (20)

Similar to 鬼勳紳梗釵喧匹插賊諾潔卞⻌曰煋手丹 (20)

More from Takashi Yoshinaga (20)

Recently uploaded (15)

鬼勳紳梗釵喧匹插賊諾潔卞⻌曰煋手丹