Raising affection is the only to way to farm gem, however it takes really long time to max one affection, so i drop it after 3 months of farming gifts. Right now I still have 20 knights left to max. Wutan maybe you should get a few 2 stars girl and repeat the process .