Episode 27: Today I’m speaking with Payne, an Indexer at The Graph. Anyone who has ever visited The Graph Discord has already met Payne – he’s a very active voice and contributor within The Graph Community. In addition to his Indexer operations, Stake Squid, Payne has been instrumental at both The Graph Foundation and Edge & Node in various roles, including QA and Indexer relations.
I’ve had the fortune of having Payne on the podcast before, as a panelist on the Subgraph Migration panel (Ep. 12) and the Curation Launch panel (Ep. 18), but this is the first time I get to speak with Payne one-on-one and ask about his background and vision for The Graph.
My conversation with Payne covers a lot of topics, including not only his ideas on keeping stake decentralized, but the mechanisms by which he thinks it could be achieved. We also talk about Curation, his experience with the new Core Dev teams (StreamingFast and Figment), and so much more.
The GRTiQ Podcast owns the copyright in and to all content, including transcripts and images, of the GRTiQ Podcast, with all rights reserved, as well our right of publicity. You are free to share and/or reference the information contained herein, including show transcripts (500-word maximum) in any media articles, personal websites, in other non-commercial articles or blog posts, or on a on-commercial personal social media account, so long as you include proper attribution (i.e., “The GRTiQ Podcast”) and link back to the appropriate URL (i.e., GRTiQ.com/podcast[episode]). We do not authorized anyone to copy any portion of the podcast content or to use the GRTiQ or GRTiQ Podcast name, image, or likeness, for any commercial purpose or use, including without limitation inclusion in any books, e-books or audiobooks, book summaries or synopses, or on any commercial websites or social media sites that either offers or promotes your products or services, or anyone else’s products or services. The content of GRTiQ Podcasts are for informational purposes only and do not constitute tax, legal, or investment advice.
We use software and some light editing to transcribe podcast episodes. Any errors, typos, or other mistakes in the show transcripts are the responsibility of GRTiQ Podcast and not our guest(s). We review and update show notes regularly, and we appreciate suggested edits – email: iQ at GRTiQ dot COM. The GRTiQ Podcast owns the copyright in and to all content, including transcripts and images, of the GRTiQ Podcast, with all rights reserved, as well our right of publicity. You are free to share and/or reference the information contained herein, including show transcripts (500-word maximum) in any media articles, personal websites, in other non-commercial articles or blog posts, or on a on-commercial personal social media account, so long as you include proper attribution (i.e., “The GRTiQ Podcast”) and link back to the appropriate URL (i.e., GRTiQ.com/podcast[episode]).
The following podcast is for informational purposes only the contents of this podcast do not constitute tax legal or investment advice, take responsibility for your own decisions, consult with the proper professionals and do your own research.
And this is the reason why I love The Graph so much because it’s so different than anything else. And it’s always, you always have something new to learn, you always find things to, to do things to improve things to work on.
Welcome to the GRTiQ podcast. Today I’m speaking with Payne and Indexer at The Graph. Anyone who has ever visited The Graph, Discord has already met Payne. He’s a very active voice and contributor to The Graph community. In addition to his Indexer operation, Stake Squid, Payne has been instrumental at both The Graph foundation and Edge & Node in various roles, including QA. I’ve had the fortune of having Payne on the podcast before as a panelist on the subgraph migration panel and on the curation launch panel. But this is the first time I get to speak with Payne one on one, and ask about his background and vision for The Graph. A Conversation with Payne covers a lot of timely topics, including not only his ideas on the topic of keeping stake, decentralized, but the mechanisms by which you would do it. We also talk about curation, his experience with the new core dev teams, and so much more. As always, we started the conversation talking about Payne’s, professional, and educational background.
I started my university in Bucharest as a an Industrial engineering that three years out of four dropped out that the fourth year didn’t quite like it. And one of the reasons why I did that was because I found crypto and blockchain. And I’ve became very interested in that and also got basically a job as a researcher in in a small company that was focused on investing. So I continued with that, or I think, almost a year. And then I started a career in trading. In the meantime, I was basically learning every single day. And yeah, I started a career in trading been on my own basically for like another year and a half. And then I was just approached by a couple of a couple of friends of mine that were running crypto caliber, which is a media company basically researching writing reviews and written research papers on crypto projects. And I’ve been with them for a while. And within that company I met Sebastian and yeah, we basically start the Stake Squid. We both had some experience, him more than I do. I started learning as fast as I could.
Where are you currently located?
I’m currently living in Romania. We’ve been here my whole life. I’ve been born and raised here. Trying to travel a bit. But yeah, with all the Corona virus stuff. It wasn’t fun lately.
What can you tell us about the people have Romania’s attitude towards crypto?
We have a big community here. I think probably one of the reasons being inflation but it’s not that crazy as compared to like, let’s say Argentina or Brazil. But there’s a lot of people that have been introduced to crypto via Elrond not sure if you know that project, but it’s a… it’s a layer one. I don’t think that’s EVM compatible. But yeah, it’s been Elrond is a Romanian project. That was they started in 2018. I was basically in one of their very first conferences. They were literally just bootstrapping that project and looking for investment back then. It was fun. So yeah, I think most of the people came in contact with crypto via them.
So what motivates somebody who’s doing research in trading in crypto to become an Indexer?
I know I’ve been all over the place which is which is a weird But it really helped because especially on research site, because I, I knew exactly what The Graph was doing. I knew exactly what it was useful for and why people would actually use something. That’s
You and your co-founder at Stake Squid. How did you first get involved in noncustodial staking services?
Yeah, so his name is Sebastian, He’s been a computer scientist for like, most of his life. And it was it was running stuff by his own because he didn’t have, I think anyone to be on his side. And yeah, it was just, it was it was learning playing around running a bunch of validators, I forgot which nodes. He also ran miners. So he definitely had tons of experience. And I saw him doing it. It was it was talking in our internal chat. And I was like, Hey, can I do this with you? And then yeah, basically, that that was the was the story.
Do you remember when you first became aware of The Graph? And what drew you to the project?
Yeah. So we found out about The Graph, a couple of months after the company was formed, we couldn’t engage in the community or in the stuff that they were building. But eventually, Sebastian found a blog post last summer in July, that basically opened up the test net for Indexers. And we also like, back in the day, we had a bunch of experience that we thought it was it was enough for us to jump in. It was definitely not. We definitely learned so much. So much new stuff. But yeah, we jumped in, we had a blast. I was I was very intrigued and hooked up by the community and by the project and by what everyone was doing. So I kept grinding and learning new stuff and helping other people understand the network, and understand how the different roles of the protocol work. And yeah, that’s kind of the story.
How did you get involved with The Graph team?
I think it was November last year, or late October. I had a DM from Yaniv, he was a said like, hey, do you have time to jump on a call? I said, yeah, why not? I never spoke with him before. But I was always keen on learning or speaking more with the founders because I was interacting on kind of daily slash weekly basis with a Janis in the chats of this score during the test net. So yeah, I jumped on a call with Yaniv. And yeah, we chatted for like half an hour. And then a couple of weeks later, another chat. And then I had my contract. So if you if you if you want to see exactly how those conversations evolved, I highly recommend watching the ETHCC recording of Yaniv’s panel, it was Yaniv and Eva talking about talking about these things in Paris. It’s a very, very interesting, very interesting panel, I highly recommend that.
And so you work on multiple networks with Stake Squid, not just The Graph here at Solano, avalanche and others. What do you do across these different networks?
So yeah, we run run validators for I believe, nine different networks or protocols. We’re currently in the works of adding two more in our portfolio. But mostly me personally, I’m mostly focused on The Graph, Solana and Agoric. Sebastian has the rest.
How is The Graph different from some of these other networks you guys are working on?
Oh, man, don’t get me started. It’s everything else is quite literally at this point. Given that those networks are using principles like quiet old on the market, stuff like Solana, and for example is literally just you plug a few commands in, you press Start and it magically works. For The Graph. We try to do something similar by running like by having our Docker compose stack that I think most of the network uses at right now, but it’s a lot more different than I don’t know how to explain it like The Graph has so many moving parts compared to like Solana for example, right. In The Graph stack, you have The Graph node, which you can split it into two the query in the index node. And then you have the Indexer agent indexes service that database or databases depending on how much You want to scale or have redundancy and stuff on this very complex system compared to Mina Solana Avalanche and that sort of stuff. And this is the reason why I love The Graph so much, because it’s so different than anything else. And it’s always, you always have something new to learn, you always find things to, to do things to improve things to, to work on. I’m not aware if you know, but recently, I’ve been working on the test net repository, I had an advanced configuration that was using as a separate branch. But now I integrated it into the main branch, and technically speaking, anyone that was running the old version before, and upgrades now, the basically seamless upgrade, you don’t need to do anything manually, just make sure that you have the right environment variables, which, which is fun. And it’s a lot more configurable than the normal stack that we were running with, for example. And we were planning to work on that going forward to make it even more robust and even to give people even more choices of configuration.
So for listeners that wouldn’t be familiar with this idea of test net repository. What is that?
It’s basically a GitHub page, where you can view instructions and download all the necessary software to basically spin up your graph node.
So how is The Graphs community different from some of the other protocols that Stake Squid works with?
This is a very good question. Well, in the case of The Graph, we have people from all the other networks that are contributing to the ecosystem that are running. That are building subgraphs, they are running their own nodes. And this is one part of the community that’s very different. Because like, yeah, I mean, on Solana Yeah, you have people building on Solana alone, but on The Graph, you have people building on BSC, you have people building on Polygon. Yeah. And it’s a much more diverse community.
So you mentioned earlier that there’s a little bit more complexity in working at The Graph than some of these other protocols. Can you explain what that complexity is and how it’s different?
Yeah, so The Graph has so many moving parts, it has to make sure that everything runs well together, and nothing breaks when whenever there’s an update, right? And the fact that there’s also a disparity between the hosted service and the network, right now is also one of the complex parts, because you have to make sure that whenever you upgrade The Graph node, that doesn’t break neither of the network’s I know, eventually, the hosting service will eventually be deprecated. But until there’s feature parity, I think that’s still we’re still going to have to deal with this struggle. The Graph is a very complex system that needs to be very carefully designed and graded so that nothing breaks. From the perspective of a validator. Again, it’s a whole architecture that you need to design and run compared to Solana which is just a single program, basically.
Payne one thing that makes you unique is the number of Indexers you run at The Graph. At the time of this recording, I believe its five Indexers, what can you tell me about running that many indexes at The Graph?
It’s a bit tricky, because like we one of the reasons why we are running is this many is that we couldn’t combine the own stake from different vesting contracts into a single one. So we’re forced basically to turn to run multiple Indexers. And this affects our profitability because each Indexer will have its own costs associated to too many transactions on the network. So the more indexes you run, the more you have to pay in operational costs per month. And it’s a tricky situations, you have to make sure that you stay profitable if you’re going to add more Indexers on chain.
Well, in addition to operating that many Indexers and being very involved in The Graph community, you also work alongside The Graph foundation. You’ve been featured on this podcast before as a panelist when The Graph launched its curation program. And then early on, we had a panel about when we talked about the migration of subgraphs from hosted service to the main net, how would you describe your role or relationship with The Graph Foundation?
Right. So my work is split between like working with the foundation and working with Edge & Node I don’t differentiate that much in between them, right. Like on the foundation, I don’t really have a clear role but then they are on Edge & Node they do help with QA. So I’m all over the place. I don’t really have one single thing that I’m doing, I’m doing lots of things whenever I’m needed. Basically, I jump on calls with the docs team. I jump on calls with designers. I but yeah, it’s I’m all over the place. I’m not really not really having one single role.
Why are you so involved in The Graph and where so many different hats?
Well, I think one of the reasons why I’m so involved, how I got into this sport is that I, immediately once I joined the test net last year in in July, was like I was I was hooked up by the tech by the community by everything. So I just spend 12 to 18 hours a day, on average, and basically just learning and being active and experimenting and doing all sorts of crazy things in the community and in the network. So that I have accumulated some, I believe, I could say, that vast experience. So there’s not too many people that know as much as I do without trying to brag too much. There’s not even a brag, but it’s just I wish that it were some like many more people than with the knowledge that I have, in a way. I don’t know how to say. So yeah. If you if you think you have the knowledge to join the Edge & Node team. Yeah, just I’m just letting you know that we’re hiring a QA and Support Engineer in the US, I believe. So yeah. If you want to apply and you need a link I can send to you in DMs over Discord.
What is the role of QA? I know you’d mentioned that in our earlier panel we did with Nina and Juan discussing the launch of curation. What does the QA role do?
Well, making sure that the product works on that involves all the all the features and all the components of the Explorer, as well as the studio. So just making sure that we don’t have any bugs, if you find some report them, make sure they’re fixed recheck after they’re supposedly fixed, just to make sure that they are in the fixed. And yeah, make sure that once those fixes, make it to production to recheck and make sure that they are again, indeed fixed. So yeah, you need to have some kind of knowledge of the protocols so you can find edge cases abuse the system and find problems before other people find them and get stuck. For example, I don’t know, delegation transactions not working or something to sing as an example.
As I mentioned paying you join me for a podcast we did a while ago when curation services went live at The Graph You Nina and Juan, join me to talk about that important event. I want to kind of switch perspectives now and put you in the role of an Indexer as we’re now a month or more past the launch of curation services, what’s your perspective of curation as an Indexer?
Man, it was chaotic in the beginning. Because everyone was seeing new subgraphs, everyone was signaling them and couple of couple of hours later un-signaling and shuffling, GRT around the protocol, which created basically a whole lot of chaos. And when I’m saying chaos is that, the reason being is that the Indexer rewards are distributed to the whole network, proportional to each subgraphs signal value on the network. And whenever you have subgraphs having 100k signal now 200k later and 10k In a day, you’ll get people Indexers to allocate to set subgraphs, and basically base condenser graphs until they sync those subgraphs up the chain head so they can close the allocations. This basically, this whole chaos was that people were seeing subgraphs that had lots of signal they were allocating to them. And then hours later, they had zero signal because like, people shuffled those GRTs into other subgraphs. And yeah, I definitely learned my lesson, I was forced to close, I think six or seven subgraphs with a so called 0x0, Proof of Indexing. So I gave up my rewards. But that was that was only for, I don’t know, less than 12 hours of waiting that between the moments that I allocated and when I close the, the allocation. But in the meantime, things have been stabilized a little bit. There’s not that much chaotic, most of the signal has been settled on promising subgraphs, for example, Sushi, and the migration partners. And the EIP-721and the other one, I forgot the name of it, which are legit subgraphs. And yeah, things are looking great again, now, hopefully, it will, will not be so volatile, again, because it’s very, very time consuming and very frustrating both as an Indexer. And probably as a Curator as well.
What’s your advice for any listener that’s either thinking about or currently involved in curation, but just still learning it?
Curation is very, very hard, because you have to both assess the probability of that subgraphs to be used in a future. And how much is that subgraph going to be used in the future because 10% of the entire query fees that that subgraph generates will go to the Curators, but they give that subgraph won’t have that many query fees, you won’t want earn anything. And also, the earlier you are on the subgraph, the more shares you can you can, you can grab. So that’s also you have to like think fast and risk getting rock bold, or stay on the sidelines and eventually see that that’s subgraph, but actually do something and you missed the train. And it’s from this perspective is very complicated. And yeah, if you if you just want if you want a more passive role without so much risk, as a Curator, you will probably be better to just set out to be the Delegator instead in my opinion.
One concept I really haven’t understood is when curation launch and you reference this just a minute ago, there were some I don’t know, fake is the right word, but there were some non-legit subgraphs. I don’t understand how that happened or why that happens. Are you able to explain that?
Yeah, sure. So that took his permission less that means anyone can take any actions in the protocol without anyone blocking you from doing so. Right? So some people so that they can basically do that. And they fork legit subgraphs with legit code, and they publish them on the network on their address and immediately signaling them. So anyone that was coming after said people would basically get a lot less shares so that those people that basically published the subgraphs could get out on a profit. And yeah, like I said, those subgraphs, the code of them, like the index data of their subgraphs, was legit because most of them in the beginning were actual clones. Forks of real production subgraphs. But obviously those weren’t deployed by the by the teams behind the subgraphs. Or who didn’t develop those up brass. There were just a bunch of bunch of kids that tried to make some GRT on the network. Yeah.
So to better understand this, if I used The Graph, as a user, I have an dApp that I pulled data and I write a subgraph. If someone forks that subgraph and deploys it and starts sending signal on it, does that mean I have to use theirs now? Or can I still use the original one I developed? What’s the friction between the two?
That’s a very good question.Yeah. So the way it works is that the code itself, when you compile it, it gets, it gets written into an IPFS hash. That’s a basic, that’s a hash of the code of the exact code, all the letters in the subgraph YAML file, all the letters in the mappings file and stuff like that, all gets compiled into a single line of 24 characters or something like that. And I, the idea is that if you don’t want to use someone else’s code, you just modify one letter, and you have a totally new subgraph, which, which hasn’t been heard of in the network, to just publish that signal on it, and it’s yours. So if you if you do that, you also have control over how you upgrade that subgraph, in the, in the event that you need more functions in it. Or if that subgraph fails, it’s only the creator of the subgraph that is able to upgrade or modify the data inside that subgraph.
So Payne, I really like your advice for listeners who might not have the comfort level or the knowledge to participate in The Graph as a Curator that they can still participate in the form of being a Delegator? If you were to give advice to listeners that are new to The Graph, or are new to delegating, what would your advice be for them?
First of all, APY doesn’t mean anything, if your Indexer doesn’t know what it’s doing. Second of all, this initialization is very important. So you should be scrolling down more than just a few that the first 10 people on the network, third of all, be on the lookout for people that are active in the community and actively helping people out. That are participating and weekly discussions. Because these are the people that you can know for a fact that they know what they’re doing. And they will perform great.
Decentralization certainly been a hot topic within The Graph community recently, there was a post in the forum by all of her about some ideas, starting some conversation about keeping stake decentralized. I had Chris Ramus on the podcast, also an Indexer at The Graph, who has a great emphasis on keeping stake decentralized. What’s your opinion on this topic?
Yeah, I’m in decentralization is one of the most important aspects of the blockchain. This is why crypto was created in the first place. So we should be aware of certain patterns that are core in many other different, different protocols and try to learn from that, and try to protect the network in a way that it doesn’t become centralized. Because if you if you only have 10, or 15, Indexers that control 80% of the network, then you don’t have anything more than just another hosted service just run by a bunch of randos. all over the globe. 10 or 15 randos. That is not just the single server, but that’s still not decentralization, right? Because half of them started working together with malicious intent. They can seriously impact not just a bunch of people in the protocol, but like whole applications that rely on set data. And this is not something that we want to have happening.
Hi this is Payne Indexer on The Graph. If my conversation with the GRTiQ podcast has been helpful to you, please consider supporting future episodes by becoming a subscriber GRTiQ does comm slash podcast for more information? That’s GRTiQ.com slash podcast. Thanks for listening.
Payne, how would you address the issue of decentralization at The Graph? I mean, I understand it’s an important topic. It’s certainly at the top of a lot of people’s minds right now, what are the mechanisms that you would want to see implemented to ensure that stake is kept decentralized?
Right? So I think first of all, you should think you should put yourself in the boots of a regular Joe, that doesn’t know or doesn’t care about decentralization, and just opens the page and sees big numbers and clicks a button on the biggest number, right? Because big number good. So I think, first of all, we should address the way that data is displayed on the Explorer right now is I think it’s ordered by self stake. So if you look on, if you if you if you open The Graphics below, you will see basically that I don’t know, the first 10, or 15 Indexers have control over like 80 or 70% of the delegation market right now. So that’s one of the first things that I would I would do is definitely not enough, are the people that are currently delegating won’t re-delegate, just because we change the number the order number in the in the UI, but at least it helps for feature Delegators that wants to take a decision. And the same forum post, I think Oliver pointed out a very nice little detail. Solana Beach, for example, has the first X number of validators hidden in a way that you have to click a button to see them. And you will see the rest of the people where you can delegate without contributing to further centralization of network, which is one of the one of the best ideas that I’ve seen so far, in the way that people display data of the network, which is really nice to see. Other things I would do, I think Chris had a great idea, which I also had a while ago, which was having some sort of dynamic delegation cap. Right now everyone in the network is limited to 16 times their own stake, basically, so that people that self-stake the 100 million GRT, you can basically eat up a quarter of the entire supply of the network in their own delegation, which is really concerning. Whether you only need four of those people to basically have the whole network centralized onto four entities, if everyone delegates to them. And one of the ideas was to have this 16 times ratio dynamic, based on how much own stake you have, the more you have, the less delegation capacity you’ll have. So that would eventually limit in a way decentralization or one of the centralization pain points. I hope. We’ll see it we’ll see if that even gets implemented by hope it does.
So I think its important Payne to emphasize the responsibility Delegators have when it comes to helping keep stake decentralized. How would you help delegates think through their role in this important issue?
Yeah. So I think delegates should keep in mind that having a lot of self-stake in the network doesn’t always mean that those Indexers actually know what they’re doing. I don’t want to I want I don’t want to call names. But yeah, this this rule is definitely applying in the current in the current time. And the idea is that you are contributing to the network more than you think so because I Delegators have strength in numbers. There aren’t so many people that know how to run Indexers. So they will be Delegators instead. And yeah, even though we do want a lot of indexes on the network, we want to have as many indexes as Delegators in the network, which is something to keep in mind. We need delegate there just to be aware of the centralization risks and take action in that way.
At the time of this recording, there’s, I believe 159-160 Indexers that The Graph as you think about the future of The Graph, and the Indexer community, how much more do you think grows? I mean, how big does that number get?
Well, we have 7 billion GRT that is not delegated or in used in the network right now. So there’s definitely room to grow. The more Indexers the better the more Delegators. The better. The more Curators the better. Because especially on the Indexer side, you the more Indexers we have, the more redundant the whole network is, if one goes down, we have like 500 more to take his place, right?
What’s been your experience, both as some of that is very technically involved in The Graph, but also running Indexers at The Graph, what’s been your experience with the new core development teams with Streaming Fast and Figment.
So I can talk about Streaming Fast, because I had tons of interactions with them. And with their funders, they’re really great guys, they’re building something that will really, really revolutionize the way that graph works and will work in the future. And I’m super excited to get their technology properly implemented into the network. I don’t know how to explain in simple terms, but it’s a difference in night and day in the way it works compared to the way The Graph node works today. And we need to find basically a middle ground in the way that these changes get implemented in The Graph node, so that it’s compatible with the current subgraphs on the network. And it’s also compatible with the network itself, and how the indexes are running it. It’s a very complex matter. It’s it’ll be a very nice year coming to us and watching those development integrated in the currently working product. And I also wanted to point out that Streaming Fast has recently open sourced their software, we’re part of it, they’re in the in, in the process of open sourcing everything. And I highly encourage it if we have any Indexers listening to the podcast to give it a shot and see how it works, because this is the future. And we better be prepared to learn how things work. So that we don’t have any surprises when it comes later. And we will be needed to run it. Same as with Erigon nodes that have gone being Ethereum client that we’re currently trying to find out if there are any bugs and things that needs to be fixed.
Payne, I want to ask you because of your technical background and involvement with The Graph, as well as Stake Squids involvement with a lot of different networks. What do you think is the long term potential or impact of The Graph?
Well, one of the reasons why we’re so happy to be involved in The Graph is that it’s always so such a dynamic environment with so potentially so many chains that someone would interact with. So I think graphs future is very bright simply because if you think of how many people are using it today, just imagine expanding to all the other chains that exists for example in Polkadot NEAR Solana and so on having all the other developers running things on The Graph, or with the help of The Graph. So my vision for The Graph is that is going to be amazing, and it’s going to be used by everyone else. And the vision of Stake Squid with is that we need to be part of the rest of the network to learn how to operate them. When it comes the time to run those nodes for The Graph as archive nodes.
If listeners want to learn more about you or the things you’re doing at Stake Squid, what’s the best way to stay in touch?
Hit me up on Discord. That’s I’m there 12 hours a day at least I always do my best to answer the ends. If not, if I don’t answer your DM, just write again. I will eventually I’m trying my best to answer within the same hour when I got the DMs but sometimes I click the I’m clicking the icons and with the thinking that I will reply in a bit and then I forget so I’m very sorry if that happened to anyone just hit me up again and reach back.
Please support this project
by becoming a subscriber!
CONTINUE THE CONVERSATION
DISCLOSURE: GRTIQ is not affiliated, associated, authorized, endorsed by, or in any other way connected with The Graph, or any of its subsidiaries or affiliates. This material has been prepared for information purposes only, and it is not intended to provide, and should not be relied upon for, tax, legal, financial, or investment advice. The content for this material is developed from sources believed to be providing accurate information. The Graph token holders should do their own research regarding individual Indexers and the risks, including objectives, charges, and expenses, associated with the purchase of GRT or the delegation of GRT.